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I. REAL PARTY IN INTEREST 

The real party in interest for this appeal and the present application is 
Xerox Corporation, by way of an Assignment recorded in the U.S. Patent and 
Trademark Office at Reel 014676, Frame 0262. 
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II. STATEMENT OF RELATED CASES 



There are no prior or pending appeals, interferences or judicial 
proceedings, known to any inventor, any attorney or agent who prepared or 
prosecuted this application or any other person who was substantively involved 
in the preparation or prosecution of this application, that may be related to, or 
that will directly affect or be directly affected by or have a bearing upon, the 
Board's decision in the pending appeal. 
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III. JURISDICTIONAL STATEMENT 

The Board has jurisdiction under 35 U.S.C. § 134(a). The Examiner 
mailed a Final Rejection on February 10, 2009, setting a three-month shortened 
statutory period for response. A Notice of Appeal and Pre- Appeal Brief 
Request for Review were filed on May 1 , 2009. The time for filing an Appeal 
Brief expires the later of two months from the filing of the Notice of Appeal, or 
one month from the mailing date of the Notice of Panel Decision if a Pre- 
Appeal Brief Request for Review is sought. Bd.R. 41.37(c) and Official Gazette 
Notice, July 12, 2005. A Notice of Panel Decision from Pre- Appeal Brief 
Review was mailed on September 8, 2009, setting a one-month period for filing 
an Appeal Brief, or the balance of the two-month time period running from the 
receipt of the Notice of Appeal, whichever is greater. The extendible period for 
filing the Appeal Brief therefore expires on October 8, 2009. This Appeal Brief 
is being timely filed on October 8, 2009. 
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VI. STATUS OF AMENDMENTS 

No Amendments After Final Rejection have been filed, 
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VII. GROUNDS OF REJECTION TO BE REVIEWED 

The following grounds of rejection are presented for review: 
1) Claims 1-22 are rejected as having been obvious under 35 U.S.C. 
§ 103(a) over Lin, U.S. Patent Application Publication No. 2002/0076103 in 
view of "Appellants' Admitted Prior Art Background section". 
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VIII. STATEMENT OF FACTS 

1 . In the rejecting the feature "determining if reclassification is required" of 
independent claims 1 and 8 under 35 U.S.C. § 103(a), the Examiner cites 
paragraph [0054] and [0059] of Lin. See February 10, 2009 Final 
Rejection page 4, lines 8-12. 

2. Lin discloses a method and apparatus for segmenting an image using a 
combination of image segmentation techniques. See Abstract. 

3. Lin discloses that micro-detection is carried out in step S101 where 
multiple scanlines of an image are buffered into memory. See paragraph 
[0053] and Fig. 6. 

4. In step S101 a judgment is made as to whether the intensity of the pixel 
under examination is significantly different than the surrounding pixels. 
See paragraph [0053] and Fig. 6. 

5. In step S 103, a macro-detection is performed where the results of the 
micro-detection step are used to identify those pixels within each 
scanline that are edges and those pixels that belong to image runs. See 
paragraph [0054]. 

6. The image type of each image run is determined based on the micro- 
detection results. See paragraph [0054] and Fig. 6. 

7. In step S 1 05, the image runs of adjacent scanlines are combined to form 
windows. See paragraph [0055] and Fig. 36. 
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8. A window is a contiguous area, of an arbitrary shape, having pixels that 
are all of the same class. See paragraph [0055]. 

9. In step S 107, statistics are gathered and calculated for each of the 
windows. The statistics are based on the intensity and macro-detection 
results of each of the pixels within the window. See paragraph [0056]. 

10. In step SI 09, the statistics are examined in an attempt to classify each 
window. Windows that appear to contain primarily a single type of 
image data are classified according to their dominant image types and 
windows containing more than one type of image are classified as mixed. 
See paragraph [0057]. 

11. In step S 1 10, the end of the first pass, the beginning point, and the image 
type of each of the windows is recorded. See paragraph [0058]. 

12. In step Sill, SI 13 and S 1 1 5, the micro-detection, macro-detection and 
window generation steps, respectively, are repeated. See paragraph 
[0059]. 

13. In step S 1 1 7, labeling of the pixels occurs where information about the 
image type and the window of each pixel is recorded. See paragraph 
[0059]. 

14. If a pixel is within a window that was classified as "mixed" during the 
first pass, the micro-detection, macro-detection and windowing steps 



performed during the second pass are used to assign an image type to the 
pixel. See paragraph [0059]. 

15. At the end of the labeling step, each pixel is labeled as a particular image 
type. See paragraph [0059]. 

16. Lin teaches not to record the macro-detection and micro-detection results 
for each pixel. See paragraph [0060]. 

17. Lin indicates that the reason for not recording the micro-detection and 
macro-detection results from the first pass for each pixel of the image is 
because it minimizes the cost of the apparatus. See paragraph [0060]. 

1 8. The Examiner asserts that Lin discloses determining if reclassification is 
required in paragraphs [0054] and [0059]. See February 10, 2009 Final 
Rejection, page 4, lines 8-12. 

19. Appellants disagree with Fact 1 8. 

20. The Examiner acknowledges that Lin does not disclose the background 
intensity level being based on substantially all of the pixels of the image. 
See February 10, 2009 Final Rejection, page 4, lines 13-17. 

21 . The Examiner asserts that 11 Appellants* admitted prior art" discloses that 
conventionally background detection is performed by sampling pixel 
values either within a sub-region of the document or across the whole 
document (i.e., being based on substantially all of the pixel values 
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(intensity level)) of the image, and cites paragraph [0012] as support. 
See Final Rejection, page 4, lines 14-17. 

22. Appellants disagree with Fact 21 . 

23. American Heritage College Dictionary, fourth edition defines "sampling" 
as "a small portion, pieces, or segment". See page 1228. 

24. The Examiner further asserts that it would have been obvious to [one of] 
ordinary skill in the art at the time when the invention was made to use 
Appellants' admitted prior art teachings to modify Lin's method by 
detecting the background based on substantially all of the pixels 1 
intensity level of the image in order to more accurately reproduce the 
image, and cites Appellants' specification paragraph [0009], lines 6-10. 
See February 10, 2009 Final Rejection, page 4, lines 13-17. 

25. Appellants disagree with Fact 24. 

26. Appellants disclose that it may be advantageous to classify pixels of 
different image types differently, and that the misclassification of a pixel 
as background can affect background suppression. See Appellants 1 
specification at paragraph [0013]. 

27. Appellants 1 Background of the Invention section discloses that to 
improve quality of reproduced images, it is determined what type of 
image is being represented by the captured image data, and that the 



13 



image data is generally stored as multiple scan lines, where each scan 
line comprises multiple pixels. See paragraph [0004]. 

28. Appellants' Background section acknowledges that it is known to 
determine an image type by separating pixels representing different types 
of images and separating the image data into windows of similar image 
types. See Appellants' specification at paragraph [0005]. 

29. Appellants' Background section acknowledges two known methods of 
classification of each pixel, the one pass method and the two pass 
method. See Appellants' specification at paragraph [0008]. 

30. Appellants acknowledge that in known two pass methods, each pixel is 
labeled in view of the information obtained after all pixels have been 
analyzed by the first pass. See Appellants' specification at paragraph 
[0008]. 

3 1 . Appellants' specification notes that in the first pass analyzing step, only a 
few scan lines of the image are analyzed. See Appellants' specification at 
paragraph [0008]. 

32. Appellants' specification indicates that the information obtained from the 
first pass for scan lines processed after the processing of a scan line 
during the first pass, is used to classify the pixels before the second pass 
during which the image data is processed, based on the determined 
classification. See Appellants' specification at paragraph [0008]. 
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33. Appellants 1 specification acknowledges that it is known to perform 
background detection by sampling pixel values either within a sub-region 
of the document (typically leading edge) or across the whole document, 
and that for conventional processes, only a portion (i.e., not the full 
document) is used to detect the background of the document to be 
reproduced. See Appellants 1 specification at paragraph [0012]. 

34. Appellants discovered that image quality is enhanced when pixels are 
classified based on substantially all of the pixels of the image. See 
Appellants 1 specification at paragraph [0012], 

35. In rejecting independent claims 1 and 8, the Final Rejection relies on 
Appellants 1 specification at paragraph [0012] as disclosing that 
background detection is performed based on substantially all of the 
pixels. Final Rejection at page 4, lines 13-17. 

36. Appellants disagree with Fact 35. 

37. In rejecting independent claim 15, the Examiner relies on similar or 
identical reasoning as the rejection of claims 1 and 8. Final Rejection, 
page 2, lines 20-21. 

38. Appellants disagree with Fact 37. 

39. Prior to this Appeal, Appellants appealed the Examiner's rejection 
alleging that Fan (U.S. Patent No. 5,850,474) discloses the background 
detection is "based on substantially all of the pixels". 
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40. An Appeal Brief was filed on July 8, 2008. 

41 . In view of the July 8, 2008 Appeal Brief, prosecution was re-opened, and 
the September 4, 2008 Office Action was issued. 

42. The September 4, 2008 Office Action contains the currently Appealed 
rejection. See September 4, 2008 rejection, page 2-5. 
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IX. ARGUMENT 

The Examiner asserts that Appellants 1 "admitted prior art background 
section" discloses the feature "determining a background intensity level of an 
image, the background intensity level being based on substantially all of the 
pixels of the image" as recited in independent claims 1 and 8, thereby allegedly 
rendering such claims obvious. February 10, 2009 Final Office Action 
(hereinafter "Final Rejection"), page 2, lines 7-21; April 14, 2009 Advisory 
Action, page 2. In response, Appellants previously pointed out how the 
Examiner has erred. November 5, 2008 Interview; November 12, 2008 
Amendment, pages 6-8; March 10, 2009 Request for Reconsideration After 
Final Rejection, pages 2-3; and May 1, 2009 Pre- Appeal Brief Request for 
Review, pages 1-4. 

The Examiner rejects "determining a background intensity level of an 
image, the background level being based on substantially all of the pixels of the 
image" recited in independent claim 1 5 for reasons similar to those explained 
above regard to claims 1 and 8 . 

The Examiner further asserts that Lin discloses the feature "determining 
if reclassification is required" as recited in independent claims 1 and 8, thereby 
allegedly rendering such claims obvious. (September 4, 2008 Rejection, page 
4, line 12-16, February 10, 2009 Final Rejection, page 4, lines 8-12). In 
response, Appellants previously pointed out how the Examiner has erred. 
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(November 12, 2008 Amendment, page 7; March 10, 2009 Request for 

Reconsideration After Final Rejection, page 3; May 1, 2009 Pre- Appeal Brief 

Request for Review, pages 3 and 4). 

The Examiner rejects "reclassifying pixels based on the results of the 

checking step" recited in claim 1 5 for reasons similar to those explained above 

regarding claims 1 and 8. Appellants explain again below why the Examiner's 

rejections of claims 1-22 for alleged obviousness over Lin in view of 

"Appellants 1 Admitted Prior Art", is erroneous. 

A. Lin In View Of 11 Appellants f Admitted Prior Art" Fails To 
Disclose or Render Obvious "The Background Intensity 
Level Being Based On Substantially All Of The Pixels Of 
The Image" Recited In Independent Claim 1 And Similarly 
Recited in Independent Claim 8 And "The Background 
Level Being Based On Substantially All Of The Pixels Of 
The Image," As Recited In Independent Claim 15 

Independent claims 1 and 8 recite, inter alia, "determining/determines a 
background intensity level of an image, . .based on substantially all of the pixels 
of the image". 

Appellants discovered that it may be advantageous to classify pixels of 
different image types differently, and that the misclassification of a pixel as 
background, for example, can affect background suppression and also the 
rendering of the types of pixels of an image. (Fact 26). In summary, 
Appellants discovered that by using only a first pass edge or other sub-region 
information and not reclassifying pixels classified as background during the 
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first pass leads to misclassification of pixels. (Fact 26), Appellants discovered 
that pixel classification is improved when the pixels are classified based on 
substantially all of the pixels of the image and then the classification of each 
pixel is confirmed. (Fact 34). 

Appellants 1 Description of Related Art section discloses that 
"background detection is performed by sampling pixel values either within a 
sub-region of the document (typically, leading end) or across the whole 
documents" (emphasis added). In particular, paragraph [0012] states; 

Conventionally, background detection is performed by sampling pixel 
values either within a sub-region of the document (typically, the leading 
edge) or across the whole document. For conventional processes, only a 
portion (i.e., not the full document) is used to detect the background of 
the document to be reproduced. The detected lead-edge or other sub- 
region background information is then used to process and classify each 
of the pixels of the scanned image. 

The use of "sampling" in Appellants 1 specification is consistent with its 
commonly understood meaning to refer to "a small portion, pieces, or segment." 
(Fact 23). This conventional method of determining intensity level of an image 
based on sampling pixel values is not. . ."a pixel classification. . .based on 
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substantially all the pixels of the image. as recited in independent claims 1 
and 8. "Sampling" is the antithesis of "substantially all," Appellants 1 
specification is consistent with this ordinary meaning of "sampling" in that 
Appellants 1 specification explicitly indicates that determining "based on 
substantially all the pixels. , ." is different from "sampling". See, for example, 
paragraphs [0014] and [0056] of Appellants' specification (especially last 
sentence of paragraph [0056]), which are reproduced below. 

Various exemplary embodiments of the invention provide a pixel 
classification method for classifying pixels of an image by determining a 
background intensity level of an image which is based on substantially 
all of the pixels of the image. The method also involves checking the 
classification of the pixel based on the determined background intensity 
level of the image. 

Various exemplary embodiments of the invention may be incorporated 
into the exemplary segmentation and processing method described 
above. In particular, various exemplary embodiments of the invention 
use the results of a full page based background detection to adjust, as 
necessary, the classification of the pixels by checking the classification. 
Various exemplary embodiments of the invention check the classification 
of a pixel by comparing the intensity of the pixel with the intensity of the 
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white point or the background intensity level of the image. The white 
point or the background intensity level of the image is determined based 
on an analysis of substantially all of the pixels of the document, and not 
just a sampling of the pixels or a sub-region of the image. 



Regarding independent claim 15, "admitted prior art" fails to disclose or 
render obvious "determining a background intensity level of an image, the 
background level being based on substantially all of the pixels of the image". 
Lin classifies the intensity level of each pixel based on the intensity of its 
surrounding pixels, not based on a background intensity level that is based on 
substantially all of the pixels of the image, (Facts 4-16). As explained above, 
the "admitted prior art" also does not disclose this feature. 

Thus, one having ordinary skill in the art would not have modified Lin in 
view of Appellants 1 "admitted prior art" to obtain the combination of features 
recited in independent claims 1, 8 and 15. 

Accordingly, Lin in view of Appellants' "admitted prior art" is clearly 
missing "the background level being based on substantially all of the pixels of 
the image". 
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B. Lin In View Of Appellants 1 Admitted Prior Art 

Fails To Disclose Or Render Obvious "Determining 
If Reclassification Is Required" As Recited In 
Independent Claims 1 And 8 And "Reclassifying 
Pixels Based On Result Of The Checking Step" 
As Recited In Independent Claim 15 

The Final Rejection asserts that Lin discloses "determining if 
reclassification is required;" and "reclassifying the pixel when reclassification 
is required." (Fact 1). Appellants respectfully disagree. 

Lin discloses a method and apparatus for segmenting using a 
combination of image segmentation techniques. (Fact 2). Lin discloses that 
during a first step S101, a micro-detection is carried out where multiple scan 
lines of an image are buffered into memory. (Fact 3). That is, only a few scan 
lines of the image are analyzed. (Fact 3). During a second step SI 03, a micro- 
detection is performed where the results of the micro-detection step are used to 
identify those pixels within each scan line that are edges in those pixels that 
belong to image runs. (Fact 5). The image type of each image run is then 
determined based on the micro-detection results. (Fact 6). In the next step 
S105, the image runs of adjacent scan lines are combined to form windows. 
(Fact 7). A window is a contiguous area in an image, of an arbitrary shape, 
where all pixels are of the same class. (Fact 8). In step SI 07, statistics are 
gathered and calculated for each of the windows. (Fact 9). In step SI 09, the 
statistics are examined in an attempt to classify each window. (Fact 10). 
However, the micro-detection and macro-detection results from the first pass 
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are not recorded for each pixel of the image, to minimize the memory 
requirements of a device embodying the invention, which renders determining 
if reclassification is required impossible. (Facts 16 and 17). 

Lin teaches away from the claims 1 and 8 feature of "determining if 
reclassification is required, , since at the time Lin makes a first pass of the 
image, Lin does not record the macro-detection or micro-detection results for 
each pixel of the image. (Fact 17). More specifically, because Lin does not 
record the macro-detection or micro-detection results for each pixel, it is 
impossible for each pixel to be reclassified, as recited in independent claims 1 
and 8. 

Additionally, the February 2, 2009 Final Rejection and April 14, 2009 
Advisory Action do not respond to Appellants 1 argument that Lin teaches away 
from the claims 1 and 8 feature of "determining if reclassification is 
required . . because at the time Lin makes a first pass of the image, Lin does 
not record the macro-detection or micro-detection results from the pixel of the 
image. Lin paragraph [0060] states: 

Once each portion of the image data has been classified according 
to standard image types, farther processing of the image data can 
be efficiently performed. Because the micro-detection and macro- 
detection results from the first pass are not recorded for each pixel 



of the image, the memory requirements for a device embodying 
the invention are minimized. This helps to minimize the cost of 
such an apparatus. 

Appellants 1 specification explicitly acknowledges systems such as Lin f s 
and distinguishes from such systems, for example, at paragraph [0013], which 
states: 

In known two-pass methods, for example, the original 
classification of a pixel as background is done during the first 
pass using lead-edge or other sub-region information and pixels 
misclassified as background during the first pass are not re- 
classified during the second pass. As lead-edge or other sub- 
region information may not be a true indication of the 
background of the captured image, misclassification of pixels as 
background can occur. For example, a background pixel can be 
classified as smooth contone or vise versa. Similarly, in known 
two-pass methods, pixels are subjected to a second pass when the 
pixel was associated with a "mixed" window during the first 
pass. Thus, in known classification methods, the classification 
of a pixel is not reconsidered. However, as discussed above, 
because it may be advantageous to classify pixels of different 
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image types differently, the misclassification of a pixel as 
background, for example, can affect background suppression and 
also the rendering of the types of pixels, 

Lin performs the pixel classification as explained in Appellants 1 
Background of the Invention section. (Facts 13-17). Thus, because Lin does 
not record the macro-detection or micro-detection results of each pixel, it is 
impossible for each pixel to be reclassified, as recited in independent claims 1 
and 8. 

Regarding independent claim 15, Lin fails to disclose or render obvious 
"checking the classification of at least portion of the pixels of the image based 
on the determined background intensity level image . . .." Lin classifies the 
intensity of each pixel based on the intensity of its surrounding pixels, not 
based on a background intensity level that is based on substantially all of the 
pixels of the image. See Lin, paragraph [0053]. As explained above, the 
"admitted prior art" does not disclose this feature. 

C. De pendent Claims 

Claims 2-7, 9-14 and 16-22 depend either directly or indirectly from one 
of independent claims 1, 8 and 15, Claims 2-7, 9-14 and 16-22 are thus 
patentable at least for their dependency from their respective independent 
claims, as well as for the additional features they recite. 
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*L* *x* *.!> *x* 

#p rj> rj> rfp* 



For all of the reasons discussed above, it is respectfully submitted that 
the rejections are in error and that claims 1-22 are in condition for allowance. 
For all of the above reasons, Appellants respectfully request this Honorable 
Board to reverse the rejection of claims 1-22. 
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Email: email@oliff.com 

Filed: October 8, 2009 



Respectfully submitted, 




James A. Oliff 
Registration No. 27,075 



Robert G, Bachner 
Registration No. 60,122 
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X. APPENDIX A - CLAIMS SECTION 

1. (Rejected) A pixel classification method, comprising: 
determining a background intensity level of an image, the background 

intensity level being based on substantially all of the pixels of the image; 
classifying a pixel of the image; 

confirming the classification of the pixel based on the determined 
background intensity level of the image by comparing the intensity of the pixel 
with the determined background intensity level; 

determining if reclassification is required; and 
reclassifying the pixel when reclassification is required. 

2. (Rejected) The pixel classification method of claim 1, wherein the 
determining step comprises determining a white point of the image based on at 
least one characteristic of substantially all of the pixels of the image. 

3. (Rejected) The pixel classification method of claim 2, wherein the 
confirming step comprises comparing the intensity of the pixel with an intensity 
of the white point of the image. 

4. (Rejected) The pixel classification method of claim 3, wherein the 
reclassifying step includes reclassifying the pixel as background when the pixel 
is classified as a class eligible to be reclassified and the intensity of the pixel is 
not less than the intensity of the white point of the image. 



5. (Rejected) The pixel classification method of claim 3, wherein the 
reclassifying step includes reclassifying the pixel as one of smooth contone and 
an equivalent class when the pixel is classified as background and the intensity 
of the pixel is less than the intensity of the white point of the image. 

6. (Rejected) The pixel classification method of claim 1, wherein the 
determining step comprises identifying a spread of intensity levels of 
substantially all the pixels of the image and determining an intensity level of a 
majority of the pixels. 

7. (Rejected) The pixel classification method of claim 4, wherein the 
pixel is classified as smooth contone. 

8. (Rejected) A pixel classification apparatus, comprising: 

a background intensity level determining module that determines a 
background intensity level of an image based on substantially all of the pixels 
of the image; and 

an image processing module that classifies a pixel of the image, confirms 
the classification of the pixel based on the determined background intensity 
level of the image by comparing the intensity of the pixel with the determined 
background intensity level, determines if reclassification is required, and 
reclassifies the pixel when reclassification is required. 

9. (Rejected) The pixel classification apparatus of claim 8, wherein 
the background intensity level determining module determines a white point of 



the image based on a characteristic of substantially all of the pixels of the 
image. 

10. (Rejected) The pixel classification apparatus of claim 9, wherein 
the image processing module confirms the classification of the pixel by 
comparing the intensity of the pixel with the intensity of the white point of the 
image. 

11. (Rejected) The pixel classification apparatus of claim 10, wherein 
when a pixel is classified as a class eligible to be reclassified and the intensity 
of the pixel is not less than the intensity of the white point of the image, the 
pixel is reclassified as background. 

12. (Rejected) The pixel classification apparatus of claim 10, wherein 
when a pixel is classified as background and the intensity of the pixel is less 
than the intensity of the white point of the image, the pixel is reclassified as 
smooth contone. 

13. (Rejected) The pixel classification apparatus of claim 8, wherein 
the image processing module identifies a spread of intensity levels of 
substantially all the pixels of the image and determines an intensity level of a 
majority of the pixels. 

14. (Rejected) The pixel classification apparatus of claim 11, wherein 
the pixel is classified as one of smooth contone and an equivalent class. 

15. (Rejected) An image processing method, comprising: 
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determining a background intensity level of an image, the background 
level being based on substantially all of the pixels of the image; 
classifying a pixel of the image; 

checking the classification of at least a portion of the pixels of the image 
based on the determined background intensity level of the image; 

reclassifying pixels based on results of the checking step; and 
processing image data of the pixels of the image based on the 
classification of the pixels. 

16. (Rejected) The image processing method of claim 1 5, further 
comprising storing a label associated with each of substantially all of the pixels, 
wherein the label of each of substantially all of the pixels is based on results of 
the classification step and the checking step for the pixel. 

17. (Rejected) The image processing method of claim 15, wherein 
classifying a pixel of the image comprises classifying the pixel as one of 
smooth contone, rough contone, text, background, graphics and halftone. 

18. (Rejected) The image processing method of claim 1 5, wherein the 
determining step comprises determining a white point of the image based on a 
characteristic of substantially all of the pixels of the image. 

19. (Rejected) The image processing method of claim 1 8, wherein the 
checking step comprises comparing an intensity of the pixel with an intensity of 
the white point of the image. 
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20. (Rejected) The image processing method of claim 19, wherein 
when the pixel is classified as smooth contone and the intensity of the pixel is 
not less than the intensity of the white point of the image, the pixel is 
reclassified as background. 

21 . (Rejected) The image processing method of claim 1 9, wherein 
when the pixel is classified as background and the intensity of the pixel is less 
than the intensity of the white point of the image, the pixel is reclassified as 
smooth contone. 

22. (Rejected) The image processing method of claim 15, wherein the 
portion of the pixels comprises substantially all of the pixels of the image. 
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XI. APPENDIX B - CLAIM SUPPORT 

AND DRAWING ANALYSIS SECTION 

1. A pixel classification method, {paragraphs [0001]; [0013]; and 
[0027]} comprising: 

determining a background intensity level of an image, {[0014]; 
[0054]-[0056]; [0058]; Figure 9, S910} the background intensity level being 
based on substantially all of the pixels of the image {[0014]-[0016]; [0056]; 
[0058]; Figure 9, S910}; 

classifying a pixel of the image {J0015]; [0016], [0027]; [0028]; [0036]; 
[0040]; [0041]-[0044]; [0046]; [0052]; and [0058]; and Figure 9, S910}; 

confirming the classification of the pixel based on the determined 
background intensity level of the image by comparing the intensity of the pixel 
with the determined background intensity level {[0014]-[0016]; [0028]; 
[0058]; [0062]; [0063]; and Figure 9, S930}; 

determining if reclassification is required {[0054]; [0058]; [0062]; 
[0063]; and Figure 9, S940}; and 

reclassifying the pixel when reclassification is required {[0016]; [0027]; 
[0054]; [0059]-[0062]; and Figure 9, S940}. 

8. A pixel classification apparatus, comprising: 

a background intensity level determining module {[0015]; [0027]; 
Figure 2 elements 70; 110 and Figure 3} that determines a background 
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intensity level of an image based on substantially all of the pixels of the image 
{[0014H0016]; [0056]; [0058], and Figure 9, S910}; and 

an image processing module {[0015]; [0027]; [0033]; [0064]; and 
Figure 2 elements 70, 110 and Figure 3} that classifies a pixel of the image 
{[0015]-[0016]; [0027]-[0028]; [0036]; [0040]-[0042]; [0044]; [0046]; [0052]; 
[0058]; and Figure 9, S920}, confirms the classification of the pixel based on 
the determined background intensity level of the image by comparing the 
intensity of the pixel with the determined background intensity level 
{[0014]-[0016]; [0028]; [0058]; [0062]-[0063]; and Figure 9, S930}, 
determines if reclassification is required {[0054]; [0058]; [0062]-[0063]; and 
Figure 9, S940}, and reclassifies the pixel when reclassification is required 
{[0016]; [0027]; [0054]; [0059]- [0062]; and Figure 9, S940}. 

15. An image processing method {[0001]; [0013]; and [0027]}, 
comprising: 

determining a background intensity level of an image {[0014]; 
[0054]-[0056]; [0058]; and Figure 9, S910}, the background level being based 
on substantially all of the pixels of the image {[0014]-[0016]; [0056]; [0058]; 
and Figure 9, S910}; 

classifying a pixel of the image {[0015]-[0016]; [0027]-[0028]; [0036]; 
[0040]-[0042]; [0044]; [0046]; [0052]; [0058]; and Figure 9, S920}; 
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checking the classification of at least a portion of the pixels of the image 
based on the determined background intensity level of the image 
{[0014H0016J; [0028]; [0058]; [0062]-[0063]]; and Figure 9, S930}; 

reclassifying pixels based on results of the checking step {[0016]; 
[0027]; [0054]; [0059]-[0062]; and Figure 9, S940}; and 

processing image data of the pixels of the image based on the 
classification of the pixels {[0016]; [0040]; [0053]; [0064]}. 
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XII. APPENDIX C - MEANS OR STEP PLUS 

FUNCTION ANALYSIS SECTION 



NONE 
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Electronic Version 
Stylesheet Version v1 .1 .1 

Description 

SYSTEJVIS AND ]VIETH[OI3S FOR 
ADJUSTING PIXEL CLASSIFICATION 
USING BACKGROUND DETECTION 

BACKGROUND OF THE INVENTION 

1. FIELD OF INVENTION 

[0001] The present Invention relates generally to pixel classification of a 

scanned image and, more particularly, to using background detection 
results for adjusting and/or determining the classification of a pixel. 

2. DESCRIPTION OF RELATED ART 

[0002] Image capture devices, such as scanners, and image forming 

devices, such as copiers, convert the light reflected from an original 
document into electrical charges that represent the light intensity of 
predetermined areas, e.g., pixels, of the document. The electrical 
charges are then processed and signals that can be used to recreate 
the captured image are generated. 

[0003] One criteria for evaluating the performance of an image capture 

device or an image forming device is how well the reproduced image 
matches the original image. To improve the quality of the reproduced 
image, multiple steps and considerations are involved during the 
processing of the captured image data. 
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[0004] For example, to improve the quality of the reproduced image, 

generally, it is determined what type of image is being represented by 
the captured image data. Image data is generally stored in the form of 
multiple scan lines, where each scan line comprises multiple pixels. 
When processing the image data, it is advantageous to know the type 
of image represented by the data because it may be advantageous to 
process each of the image types differently. The image data can 
represent various types of images including, for example, graphics, 
text, background, smooth continuous tone (smooth contone), rough 
continuous tone (rough contone), and halftones of different 
frequencies. Further, a page of image data can be a single image type 
or some combination of image types. 

[0005] 

To determine the type of image being represented by a pixel and to 
separate pixels representing different types of images, it is known, for 
example, to take a page of image data and to separate the image 
data into windows of similar image types. A page of image data may, 
for example, include a halftoned picture with accompanying text 
describing the picture. To efficiently process the image data, the page 
of the image data may be separated into windows such that a first 
window represents the halftoned image and the second window 
represents the text. Processing of the image data is then carried out 
by customizing the processing based on the type of image data being 
processed in order to improve the quality of the reproduced image. 
For example, the image data may be subjected to different filtering 
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mechanisms based on the determined type of the image data. 

[0006] Accordingly, in order to improve the quality of the reproduced image, it 
is important for image data to be classified correctly. If the image data 
is not classified correctly, inappropriate processing may actually 
diminish the quality of the image data and the reproduced image. 

[0007] When classifying each pixel individually or when grouping the image 
data such that each group of pixels (e.g., a window) represents a 
different type of image data, generally, it is known to make either one 
or two passes through the page of image data. 

[0008] In the one pass method, the classification of the pixel is based on the 
information obtained regarding the pixel during a single pass through 
the image data, and thus, processing is performed "on the fly" such 
that pixels are classified after only one or a few scan lines are 
analyzed. On the other hand, in the two pass method, each pixel is 
processed and labeled in view of the information obtained after all the 
pixels have been analyzed. More particularly, in the two pass method, 
information obtained from the first pass for scan lines processed after 
the processing of a scan line during the first pass is used to classify 
the pixels before the second pass during which the image data is 
processed, based on the determined classifications. For example, in 
the two pass method, information obtained for a subsequent scan line 
can be used to generate or correct information for a previous scan 
line. In some two pass methods, two rounds of pixel level analysis are 
performed on all the pixels before the pixels are classified while in 
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other two pass methods a single round of pixel level analysis (i.e., a 
single run through the pixels of the image) is performed before the 
pixels are classified. U.S. Patent No. 5,850,474, the entire disclosure 
of which is incorporated herein by reference, discloses an example of 
such a two-pass method. 

[0009] Another example of a step which may be carried out to improve the 
quality of the reproduced image is determining the contrast of the 
original image. The contrast of the original image is determined before 
the captured image data is processed and the determined contrast is 
used to process the image data. Background detection processes are 
helpful for determining the contrast of an image. By determining the 
background of the original document, the background of the captured 
image can be used to more accurately reproduce the image. 

[0010] Generally, background detection processes collect light intensity 
information and use the collected light intensity information to 
determine an intensity level associated with the document 
background. The determined intensity level is also referred to as the 
"background intensity level". Using the image data of the captured 
image, statistical analysis, generally a histogram, can reveal a peak 
which identifies the intensity of a majority of the pixels. The peak may 
be referred to as a white-peak, a white point or a background peak. 
The white peak, for example, is the gray level with the greatest 
number of pixels having an intensity related to the white background 
of the scanned image. 
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[001 1] The histogram is also used to determine the gain factor for the 

document. The gain factor is used to compensate for the background 
gray level of the image of the scanned document. It should be noted, 
however, that although the histogram assists in the determination of 
the background value for the document (page), the background value 
is only as accurate as the created histogram and the identified peak of 
the histogram on which it is based. 

[0012] Conventionally, background detection is performed by sampling pixel 
values either within a sub-region of the document (typically, the 
leading edge) or across the whole document. For conventional 
processes, only a portion (i.e., not the full document) is used to detect 
the background of the document to be reproduced. The detected lead- 
edge or other sub-region background information is then used to 
process and classify each of the pixels of the scanned image. 

SUMMARY OF THE INVENTION 

[0013] 

In known two-pass methods, for example, the original classification of 
a pixel as background is done during the first pass using lead-edge or 
other sub-region information and pixels classified as background 
during the first pass are not re-classified during the second pass. As 
lead-edge or other sub-region information may not be a true indication 
of the background of the captured image, misclassification of pixels as 
background can occur. For example, a background pixel can be 
classified as smooth contone or vise versa. Similarly, in known two- 
pass methods, pixels are subjected to a second pass when the pixel 
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was associated with a "mixed" window during the first pass. Thus, in 
known classification methods, the classification of a pixel is not 
reconsidered. However, as discussed above, because it may be 
advantageous to classify pixels of different image types differently, the 
misclassification of a pixel as background, for example, can affect 
background suppression and also the rendering of the types of pixels. 

[0014] Various exemplary embodiments of the invention provide a pixel 

classification method for classifying pixels of an image by determining 
a background intensity level of an image which is based on 
substantially all of the pixels of the image. The method also involves 
checking the classification of the pixel based on the determined 
background intensity level of the image. 

[0015] Various exemplary embodiments of the invention separately provide a 
pixel classification apparatus. The pixel classification apparatus 
includes a background intensity level determining module which 
determines a background intensity level of an image based on 
substantially all of the pixels of the image. The pixel classification 
apparatus also includes an image processing module which classifies 
a pixel of the image, and checks the classification of the pixel based 
on the determined background intensity level of the image. 

[0016] Various exemplary embodiments of the invention separately provide 
an image processing method. The image processing method 
determines a background level of an image, based on substantially all 
of the pixels of the image. The image processing method also 
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classifies a pixel of the image, checks the classification of the pixel 
based on the determined background intensity level of the image, 
reclassifies pixels based on the results of the checking step and 
processes image data based on the classification of the pixel. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0017] Various exemplary embodiments of systems and methods of the 
invention will be described in detail, with reference to the following 
figures. 

[0018] Fig. 1 is a diagram illustrating components of an exemplary digital 
scanner. 

[0019] Fig. 2 is a block diagram illustrating the electronic architecture of an 
exemplary digital scanner coupled to a workstation, a network, a 
storage medium and an image output terminal in accordance with 
various exemplary embodiments of the invention. 

[0020] Fig. 3 is a block diagram illustrating an exemplary architecture of an 
image processing module. 

[0021] Fig. 4 shows an exemplary two-dimensional look-up table which may 
be used to classify image data. 

[0022] Fig. 5 is a block diagram of another exemplary embodiment of an 
image segmentation module. 

[0023] Fig. 6 is a flowchart outlining an exemplary two pass segmentation 
and classification method. 
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[0024] Fig. 7 shows a graphical representation of a scan line of image data. 

[0025] Fig. 8 shows a graphical representation of scan lines of image data 
that have been separated into windows. 

[0026] Fig. 9 is a flowchart outlining an exemplary method for classifying 
pixels of an image. 

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 

[0027] The invention generally relates to methods and systems for adjusting, 
as necessary, and/or determining the classification of the pixels of a 
document based on full-page background detection results during 
capture and out of an image, for example, by a digital scanner. Such a 
digital scanner is capable of being connected to a wide array of 
copiers, printers, computers, networks, facsimile machines, and the 
like, and is capable of scanning and producing complex and 
interesting images to be stored, printed and/or displayed. The images 
may include text, graphics, and/or scanned or computer-generated 
images. With such a scanner, high quality image output can be 
achieved by automatically determining an image background based 
on the results of a full-page background detection process and using 
the image background to dynamically adjust/reclassify, as necessary, 
or more accurately determine, the classification of a pixel. 

[0028] 

It should be understood that various exemplary embodiments of the 
invention may be used in conjunction with any known pixel 
classification method in order to adjust, confirm and/or determine the 
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classification of a pixel by using the results of a full-page background 
detection process for the document. However, for purposes of 
illustration, exemplary embodiments of classification and/or 
segmentation processes are described below. Various exemplary 
embodiments of the invention may be used to adjust and/or confirm 
the classification of pixels obtained using, for example, the methods 
described below. 

[0029] Fig. 1 illustrates components of an exemplary scanning unit 20 of a 
digital scanner. In the scanning unit 20, a light source 21 is used to 
illuminate a document 22 to be scanned. In a platen-type scanning 
situation, the document 22 usually rests upon a glass platen 24, which 
supports the document 22 for scanning purposes. The document 22 
may be placed on the glass platen 24 by an operator. Alternatively, 
the scanning unit may include a feeder or document handler 29, which 
places the document on the glass 24. 

[0030] 0n ^ of the glass platen 24 and the document 22, a backdrop 

portion (platen cover) 26 is placed to prevent stray light from leaving 
the scanning area to provide a background from which an input 
document can be distinguished. The backdrop portion 26 may be part 
of document handler 29. The backdrop portion 26 is the surface or 
surfaces that can be scanned by an image-sensing unit 28 when a 
document is or is not present in the scanning station. The light 
reflected from the document passes through a lens subsystem (not 
shown) so that the reflected light impinges upon the image sensing 
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unit 28, such as a charge coupled device (CCD) array or a full width 
array. 

[0031] A full width array typically comprises one or more linear arrays of 
photo-sites, wherein each linear array may be sensitive to one or 
more colors. In a color image capture device, the linear arrays of 
photo-sites are used to produce electrical signals which are converted 
to color image data representing the scanned document. However, in 
a black-and-white scanner, preferably, only one linear array of photo- 
sites is used to produce the electrical signals that are converted to 
black and white image data representing the image of the scanned 
document. 

[0032] Fig. 2 is a block diagram illustrating the electronic architecture of an 
exemplary digital scanner 30 including the scanning unit 20. The 
digital scanner 30 is coupled to a workstation 50 by way of a scanning 
interface 40. An example of a suitable scanning interface is a SCSI 
interface. Examples of the workstation 50 include a personal computer 
or a computer terminal. The workstation 50 includes and/or has 
access to a storage medium 52. The workstation 50 may be adapted 
to communicate with a computer network 54, and/or to communicate 
with the Internet either directly or through the computer network 54. 
The digital scanner 30 may be coupled to at least one image output 
terminal (IOT) 60, such as a printing system, via the workstation 50, 
for example. 

[0033] The scanning unit 20 scans an image and converts the analog signals 
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received by the image sensing unit 28 into digital signals (digital data). 
An image processing unit 70 registers each image, and may execute 
signal correction to enhance the digital signals. As the image 
processing unit 70 continuously processes the data, a first-in first-out 
(FIFO) buffer 75 temporarily stores the digital data output by the 
image processing unit 70, and transmits the digital data, for example, 
to the International Telecommunications Union (ITU) G3/G4 80 and 
Joint Photographic Experts Group (JPEG) 85 in bursts, so that the 
processed data is compressed. Other data compression units may be 
substituted for the ITU G3/G4 80 and the JPEG 85. The compressed 
digital data is stored in a memory 100, for example, by way of a 
Peripheral Component Interconnect Direct Memory Access 
(PCI/DMA) Controller 90 and a video bus 95. Alternatively, an 
operator may not wish to compress the digital data. The operator may 
bypass the compression step so that the data processed by the image 
processing unit 70 is sent through FIFO 75 and directly stored in the 
memory 100 by way of the PCI DMA controller 90. 

[0034] 

A computing unit 110, such as a microprocessor, is coupled to the 
scanner interface 40, the memory 100 and the PCI DMA controller 90 
by way of the video bus 95 and a video bus bridge 120. The 
computing unit 110 is also coupled to a flash memory 130, a static 
RAM 140 and a display 150. The computing unit 110 communicates 
with the scanning unit 20 and the image processing unit 70, for 
example, by way of a control/data bus. For example, the computing 
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unit 1 1 0 may communicate with the image processing unit 70 through 
the video bus 95 and/or the PCI DMA controller 90. Alternatively, the 
computing unit 110 may communicate directly with different 
components, such as the image processing unit 70 by way of 
control/data buses (not shown). 

[0035] Fig. 3 shows an exemplary architecture of an image segmentation 
apparatus 300 which may form part of the image processing unit 70 
shown in Fig. 2. 

[0036] Fig. 3 shows two exemplary features that may be extracted and used 
for image processing and/or segmentation in order to improve the 
quality of the reproduced image. The two features are video 
peak/valley count within a window containing the pixel being classified 
and local roughness. 

[0037] Local roughness may represent the degree of gray level discontinuity 
computed as a combination of some gradient operators. One example 
of local roughness is the difference between the maximum and 
minimum of nine 3x3 window sums within a 5 x 5 video context. It 
should be understood that various exemplary embodiments of the 
invention may be used in conjunction with any known or hereafter 
developed methods of determining the local roughness. 

[0038] 

On the other hand, a pixel may be considered as a video peak or 
video valley, respectively, if its gray level is the highest or the lowest in 
a neighborhood and the gray level difference between the gray level 
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of the pixel and the gray level of the neighborhood average is greater 
than a certain threshold. It should be understood that various 
exemplary embodiments of the invention may be used in conjunction 
with any known or hereafter developed methods for determining video 
peaks and/or video valleys. 

[0039] Several lines of peak and valley patterns may be recorded in scan line 
buffers for computing peak/valley count within a defined window. For 
example, various exemplary embodiments of the invention may be 
used in a system where the peak/valley count and local roughness 
are used as indices to form a two-dimensional look-up table (hereafter 
also called a classification table) as a basis to classify image data. 

[0040] FIG. 4 shows an example of a two-dimensional look up table that uses 
five roughness levels and twelve peak/valley count levels. As a result, 
the look up table includes sixty classification table entries (i.e., 
5x12=60). Depending on a location within the look-up table, the video 
data may be mapped to certain classifications such as low frequency 
halftone, high frequency halftone, smooth continuous tone, rough 
continuous tone, edge, text on halftone, and the like. Depending on 
the classification, the input data may be processed differently. For 
example, different filters may be applied, based on the classification, 
during processing of the data in order to improve the overall quality of 
the reproduced image. 

[0041] Various exemplary embodiments of the invention may be used in 

conjunction with a system in which the look-up table (i.e., classification 
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table) is complemented with some special classifications. One 
example of a possible special classification is the "edge classification". 
The "edge classification" tries to identify some line art and kanji area 
that could be missed by the look-up table. Another example of a 
special classification is the "white classification ". The "white 
classification " makes use of the absolute gray level information in 
addition to peak/valley count and roughness. A "default classification " 
may be used for the borders of an image. The classification look-up 
table output may be multiplexed with the special classification to 
produce the final classification of a pixel (i.e., classification output). 
The classification table assignment may be programmable to allow for 
more flexibility in rendering adjustment. 

[0042] 

Fig. 5 shows a block diagram of a page segmentation and 
classification apparatus 500 as another example of a portion of the 
image processing unit 70 shown in Fig. 2. The page segmentation 
and classification apparatus 500 performs a two-pass segmentation 
and classification method. The page segmentation and classification 
apparatus 500 includes micro-detection means 520 for performing a 
micro-detection step, macro-detection means 530 for performing a 
macro-detection step and windowing means 540 for grouping image 
runs of the scan lines together to form windows. The apparatus 500 
also includes statistics means 550 for gathering and calculating 
statistics regarding the pixels within each window and classification 
means 560 for classifying each of the windows as a particular image 
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type based on the gathered statistics. 

[0043] Memory means 570 is provided for recording the beginning points and 
image types of each of the windows and the beginning points and 
image types of any initially unknown image runs that are subsequently 
classified during the first pass. The memory means 570 may also be 
used to store the window and image type of each pixel at the end of 
the second pass. Typically, however, the image data is used 
immediately to process, transmit and/or print the image, and the 
image data is then discarded. 

[0044] Fig. 6 shows a block diagram illustrating an exemplary two pass 
segmentation and classification method which may be performed 
using the apparatus 500 shown in Fig. 5. The two pass segmentation 
and classification method may be used in conjunction with various 
exemplary embodiments of the invention. The method segments a 
page of image data into windows, classifies the image data within 
each window as a particular image type and records information 
regarding the window and image type of each pixel. Once the image 
type for each window and/or pixel is known, further processing of the 
image data can be efficiently performed during the second pass. For 
example, during the second pass, when the image data is being 
processed, different filters may be applied to different pixel classes in 
order to improve the quality of the reproduced image. 

[0045] As discussed above, the image data comprises multiple scan lines of 
pixel image data and each scan line typically includes intensity 
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information for each pixel within the scan line. Typical image types 
include graphics, text, low-frequency halftone, high frequency 
contone, and the like. 

[0046] Control begins in step S100 and continues to step S107. In step 

S101, micro-detection is carried out. During micro-detection, multiple 
scan lines of image data are buffered into memory. Each pixel is 
examined and a preliminary determination is made as to the image 
type of the pixel. In addition, the intensity of each pixel is compared to 
the intensity of its surrounding neighboring pixels. A judgment is made 
as to whether the intensity of the pixel under examination is 
significantly different than the intensity of the surrounding neighboring 
pixels. When a pixel has a significantly different intensity than its 
neighboring surrounding pixels, the pixel is classified as an edge 
pixel. 

[0047] 

Next in step S103, macro-detection is performed. The results of the 
micro-detection step are used to identify those pixels within each scan 
line that are edges and those pixels that belong to image runs. The 
image type of each image run is then determined based on the micro- 
detection results. The image type of an image run may also be based 
on the image type and a confidence factor of an adjacent image run of 
a previous scan line. If information obtained during an image run of a 
previous scan line is not sufficient to classify the image run as a 
standard image type, but information generated during examination of 
the current scan line makes it possible to determine the image type of 
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the image run of the previous scan line, the determination of the 
image type of that image run is made. The image type of the image 
run of the previous scan line is then recorded. 

[0048] An example of a single scan line of image data is shown in Fig. 7. 

During the macro-detection step, adjacent pixels having significantly 
different intensities from each other are classified as edges 754, 758 
and 762. Portions of the scan line between the edges are classified as 
image runs 752, 756, 760 and 764. It should be understood that 
although the micro-detection step S101 and the macro-detection step 
S103 of the exemplary segmentation method are shown sequentially, 
it is possible to carry out the steps simultaneously. 

[0049] Next in step S105, the image runs of adjacent scan lines are 

combined to form windows. It should be understood that the term 
windows may be applied to portions of the scanned image which 
contain similarly classified pixels or portions of the obtain image which 
are connected. A graphical representation of multiple scan lines that 
have been grouped into windows is shown in Fig. 8. The image data 
has been separated into a first window 812 and a second window 
813, separated by a gutter 81 1 . A first edge 814 separates the first 
window 812 from the remainder of the image data. A second edge 
816 separates the second window 813 from the remainder of the 
image data. In addition, a third edge 818 separates the second 
window 813 into a first portion 824 and a second portion 826 each 
having different image types. 
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[0050] Next in step S107, statistics are gathered and calculated for each of 
the windows and the pixels of the scanned image. The statistics are 
based on the intensity and macro-detection results for each of the 
pixels within a window. 

[0051] Next in step S109, the statistics are examined to classify each window 
and each pixel of the scanned image. 

[0052] At the end of the first pass, in step S1 1 1 , the beginning point and the 
image type of each of the windows and/or the classification tag of 
each pixel are recorded. 

[0053] Next in step S1 13, the pixels classifications are used to process the 
image data accordingly. For example, during processing of the image 
data, different filters may be applied to the data based on the 
classification of the pixel being processed. Control proceeds to step 
S1 15 where the process ends. 

[0054] 

As discussed above, various exemplary embodiments of the invention 
may be used in conjunction with any known or hereafter developed 
image segmentation and/or pixel classification systems and methods, 
such as, the exemplary systems and methods described above. 
Irrespective of the system or method used, each pixel of a scanned 
image is generally classified into one of several types of classes, such 
as, text, background, smooth contone, rough contone, halftones of 
different frequencies, and the like. Various exemplary embodiments of 
the invention use full-page background detection results to challenge 
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the classification of a pixel and to adjust/reclassify, as necessary, the 
classification of the pixel. 

[0055] It should be understood that preferably, in various exemplary 

embodiments of the invention, the full-page background detection 
results may be used to check the classification of a pixel prior to the 
labeling of the pixel. 

[0056] Various exemplary embodiments of the invention may be incorporated 
into the exemplary segmentation and processing method described 
above. In particular, various exemplary embodiments of the invention 
use the results of a full page based background detection to adjust, as 
necessary, the classification of the pixels by checking the 
classification. Various exemplary embodiments of the invention check 
the classification of a pixel by comparing the intensity of the pixel with 
the intensity of the white point or the background intensity level of the 
image. The white point or the background intensity level of the image 
is determined based on an analysis of substantially all of the pixels of 
the document, and not just a sampling of the pixels or a sub-region of 
the image. 

[0057] Fig. 9 is a flowchart outlining an exemplary method for classifying 
pixels of an image. It should be understood that although the steps 
are illustrated sequentially, the various steps may occur 
simultaneously and/or in any order. 

[0058] 

Control begins in step S900 and continues to step S910. In step 
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S910, the background intensity level of the image is determined. As 
discussed above, the background intensity level is based on 
substantially all of the pixels of the image. Next, in step S920, the 
pixels of the image are classified. Then, in step S930, the 
classification of each pixel is checked based on the determined 
background intensity level of the image. More particularly, in step 
S930 the classification of pixels classified as a pixel class eligible for 
reclassification, such as smooth contone and background, are 
checked. 

[0059] When, for example, the intensity of a pixel classified as background is 
less than the intensity of a determined white point of the image, the 
pixel is reclassified as smooth contone in step S940. Conversely, 
when, for example, the intensity of a pixel is classified as background 
is not less than the intensity of a determined white point of the image, 
the pixel's classification is confirmed as background and is not 
modified. 

[0060] When, for example, the intensity of a pixel classified as smooth 

contone is not less than the intensity of the white point of the image, 
the pixel is reclassified as background in S940. Conversely, when, for 
example, the intensity of a pixel is classified as smooth contone is less 
than the intensity of a determined white point of the image, the pixel's 
classification is confirmed as smooth contone and is not modified. 

[0061] In various exemplary embodiments of the invention, various contone 
based classes, such as, rough contone and smooth contone, are 
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eligible for classification and can be subjected to re-classification 
based on the background detection results of the scanned image. 

[0062] In some exemplary embodiments of the invention that are used in 
conjunction with systems and methods where micro-level 
classification is followed by macro-level classification (i.e., for 
example, image objects or "windows" are identified and classified, as 
described above), the classification of any or all of the pixels in both 
the non-window and window areas may be checked and adjusted, if 
necessary. That is, the results of a full page based background 
detection may be used to adjust, as necessary, the classification of 
any and/or all of the pixels. 

[0063] In various exemplary embodiments of the invention, full-page based 
background detection results are used to check/adjust the 
classification of pixels for monochrome images and/or color images. 
Various exemplary embodiments of the invention provide a method for 
classifying pixels in which misclassification of a pixel can be 
substantially and preferably, completely eliminated. For example, the 
misclassification of a pixel as a background pixel instead of a smooth 
contone may be substantially and preferably, completely eliminated. 

[0064] It should be understood that the computing unit 110, may be any 
known system capable of processing the data, such as, a special 
purpose computer, a programmed microprocessor or micro-controller 
and peripheral integrated circuit elements, an ASIC or other integrated 
circuit, a hardwired electronic or logic circuit such as a discrete 
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element circuit, a programmable logic device such as a PLD, PLA, 
FPGA or PAL, or the like. Specific algorithms may also be 
accomplished using software in combination with specific hardware. 

[0065] While the invention has been described with reference to various 
exemplary embodiments disclosed above, various alternatives, 
modifications, variations, improvements and/or substantial 
equivalents, whether known or that are or may be presently 
unforeseen, may become apparent upon reviewing the foregoing 
disclosure. Accordingly, the exemplary embodiments of the invention, 
as set forth above, are intended to be illustrative, not limiting. Various 
changes may be made without departing from the spirit and scope of 
the invention. 
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Claims 



[d] A pixel classification method, comprising: 

determining a background intensity level of an image, the 

background intensity level being based on substantially all 

of the pixels of the image; 

classifying a pixel of the image; and 

checking the classification of the pixel based on the 

determined background intensity level of the image. 

[c2] The pixel classification method of claim 1 , wherein the 

determining step comprises determining a white point of the 
image based on at least one characteristic of substantially all of 
the pixels of the image. 

[c3] The pixel classification method of claim 2, wherein the checking 

step comprises comparing an intensity of the pixel with an 
intensity of the white point of the image. 

[c4] The pixel classification method of claim 3, further comprising 

reclassifying the pixel as background when the pixel is classified 
as a class eligible to be reclassified and the intensity of the pixel 
is not less than the intensity of the white point of the image. 

[c5] The pixel classification method of claim 3, further comprising 

reclassifying the pixel as one of smooth contone and an 
equivalent class when the pixel is classified as background and 
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the intensity of the pixel is less than the intensity of the white 
point of the image. 

[c6] The pixel classification method of claim 1 , wherein the 

determining step comprises identifies a spread of intensity levels 
of substantially all the pixels of the image and determining an 
intensity level of a majority of the pixels. 

[c7] The pixel classification method of claim 4, wherein the pixel is 

classified as smooth contone. 

[c8] A pixel classification apparatus, comprising: 

a background intensity level determining module that 
determines a background intensity level of an image based 
on substantially all of the pixels of the image; and 
an image processing module that classifies a pixel of the 
image, and checks the classification of the pixel based on 
the determined background intensity level of the image. 

[c9] The pixel classification apparatus of claim 8, wherein the 

background intensity level determining module determines a 
white point of the image based on a characteristic of substantially 
all of the pixels of the image. 

[c10] The pixel classification apparatus of claim 9, wherein the image 

processing module checks the classification of the pixel by 
comparing the intensity of the pixel with the intensity of the white 
point of the image. 
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[d 1] The pixel classification apparatus of claim 10, wherein when a 

* 

pixel is classified as a class eligible to be reclassified and the 
intensity of the pixel is not less than the intensity of the white 
point of the image, the pixel is reclassified as background. 

[c12] The pixel classification apparatus of claim 10, wherein when a 

pixel is classified as background and the intensity of the pixel is 
less than the intensity of the white point of the image, the pixel is 
reclassified as smooth contone. 

[d 3] The pixel classification apparatus of claim 1 , wherein the image 

processing module identifies a spread of intensity levels of 
substantially all the pixels of the image and determines an 
intensity level of a majority of the pixels. 

[d 4] The pixel classification apparatus of claim 1 1 , wherein the pixel is 

classified as one of smooth contone and an equivalent class. 

[c15] An image processing method, comprising: 

determining a background intensity level of an image, the 
background level being based on substantially all of the 
pixels of the image; 
classifying a pixel of the image; 

checking the classification of at least a portion of the pixels 
of the image based on the determined background intensity 
level of the image; 

reclassifying pixels based on results of the checking step; 
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and 

processing image data of the pixels of the image based on 
the classification of the pixels. 

[c16] The image processing method of claim 15, further comprising 

storing a label associated with each of substantially all of the 
pixels, wherein the label of each of substantially all of the pixels 
is based on results of the classification step and the checking 
step for the pixel. 

[c17] The image processing method of claim 15, wherein classifying a 

pixel of the image comprises classifying the pixel as one of 
smooth contone, rough contone, text, background, graphics and 
halftone. 

[c18] The image processing method of claim 15, wherein the 

determining step comprises determining a white point of the 
image based on a characteristic of substantially all of the pixels 
of the image. 

[c19] The image processing method of claim 18, wherein the checking 

step comprises comparing an intensity of the pixel with an 
intensity of the white point of the image. 

[c20] The image processing method of claim 19, wherein when the 

pixel is classified as smooth contone and the intensity of the pixel 
is not less than the intensity of the white point of the image, the 
pixel is reclassified as background. 
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[c21] The image processing method of claim 19, wherein when the 

pixel is classified as background and the intensity of the pixel is 
less than the intensity of the white point of the image, the pixel is 
reclassified as smooth contone. 

[c22] The image processing method of claim 15, wherein the portion of 

the pixels comprises substantially all of the pixels of the image. 
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SYSTEMS AND METHODS FOR 
ADJUSTING PIXEL CLASSIFICATION 
USING BACKGROUND DETECTION 

Abstract 



A pixel classification method for classifying pixels of an image by 
determining a background intensity level of an image which is based 
on substantially all of the pixels of the image. The method also 
involves classifying the pixels of the image and checking the 
classification of the pixel based on the determined background 
intensity level of the image. 
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FIG. 5 
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FIG. 9 
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Sir: 

In reply to the September 4, 2008 Office Action, please consider the following: 
Amendments to the Claims as reflected in the listing of claims; and 

I 

Remarks. 
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Amendments to the Claims : 

The following listing of claims will replace all prior versions, and listings, of claims in 
the application: 

1 . (Currently Amended) A pixel classification method, comprising: 
determining a background intensity level of an image, the background intensity level 

being based on substantially all of the pixels of the image; 

classifying a pixel of the imag e without adjusting an intensity of the pixel ; 

confirming the classification of the pixel based on the determined background 
intensity level of the image by comparing the intensity of the pixel with the determined 
background intensity level; 

determining if reclassification is required; and 

reclassifying the pixel when reclassification is required. 

2. (Original) The pixel classification method of claim 1, wherein the determining 
step comprises determining a white point of the image based on at least one characteristic of 
substantially all of the pixels of the image. 

3. (Previously Presented) The pixel classification method of claim 2, wherein the 
confirming step comprises comparing the intensity of the pixel with an intensity of the white 
point of the image. 

4. (Previously Presented) The pixel classification method of claim 3, wherein the 
reclassifying step includes reclassifying the pixel as background when the pixel is classified 
as a class eligible to be reclassified and the intensity of the pixel is not less than the intensity 
of the white point of the image. 

5. (Previously Presented) The pixel classification method of claim 3, wherein the 
reclassifying step includes reclassifying the pixel as one of smooth contone and an equivalent 
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class when the pixel is classified as background and the intensity of the pixel is less than the 
intensity of the white point of the image. 

6. (Previously Presented) The pixel classification method of claim 1, wherein the 
determining step comprises identifying a spread of intensity levels of substantially all the 
pixels of the image and determining an intensity level of a majority of the pixels. 

7. (Original) The pixel classification method of claim 4, wherein the pixel is 
classified as smooth contone. 

8. (Currently Amended) A pixel classification apparatus, comprising: 

a background intensity level determining module that determines a background 
intensity level of an image based on substantially all of the pixels of the image; and 

an image processing module that classifies a pixel of the image . image without 
adjusting an intensity of the pixel, confirms the classification of the pixel based on the 
determined background intensity level of the image by comparing the intensity of the pixel 
with the determined background intensity level, determines if reclassification is required, and 
reclassifies the pixel when reclassification is required. 

9. (Original) The pixel classification apparatus of claim 8, wherein the 
background intensity level determining module determines a white point of the image based 
on a characteristic of substantially all of the pixels of the image. 

10. (Previously Presented) The pixel classification apparatus of claim 9, wherein 
the image processing module confirms the classification of the pixel by comparing the 
intensity of the pixel with the intensity of the white point of the image. 

1 1 . (Original) The pixel classification apparatus of claim 10, wherein when a 
pixel is classified as a class eligible to be reclassified and the intensity of the pixel is not less 
than the intensity of the white point of the image, the pixel is reclassified as background. 
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12. (Original) The pixel classification apparatus of claim 10, wherein when a 
pixel is classified as background and the intensity of the pixel is less than the intensity of the 
white point of the image, the pixel is reclassified as smooth contone. 

13. (Previously Presented) The pixel classification apparatus of claim 8, wherein 
the image processing module identifies a spread of intensity levels of substantially all the 
pixels of the image and determines an intensity level of a majority of the pixels. 

14. (Original) The pixel classification apparatus of claim 1 1 , wherein the pixel is 
classified as one of smooth contone and an equivalent class. 

15. (Original) An image processing method, comprising: 

determining a background intensity level of an image, the background level being 
based on substantially all of the pixels of the image; 
classifying a pixel of the image; 

checking the classification of at least a portion of the pixels of the image based on the 
determined background intensity level of the image; 

reclassifying pixels based on results of the checking step; and 

processing image data of the pixels of the image based on the classification of the 

pixels. 

16. (Original) The image processing method of claim 15, further comprising 
storing a label associated with each of substantially all of the pixels, wherein the label of each 
of substantially all of the pixels is based on results of the classification step and the checking 
step for the pixel. 

17. (Original) The image processing method of claim 15, wherein classifying a 
pixel of the image comprises classifying the pixel as one of smooth contone, rough contone, 
text, background, graphics and halftone. 

__ 4 _ 
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1 8. (Original) The image processing method of claim 1 5, wherein the determining 
step comprises determining a white point of the image based on a characteristic of 
substantially all of the pixels of the image. 

19. (Original) The image processing method of claim 18, wherein the checking 
step comprises comparing an intensity of the pixel with an intensity of the white point of the 
image. 

20. (Original) The image processing method of claim 19, wherein when the pixel 
is classified as smooth contone and the intensity of the pixel is not less than the intensity of 
the white point of the image, the pixel is reclassified as background. 

2 1 . (Original) The image processing method of claim 1 9 , wherein when the pixel 
is classified as background and the intensity of the pixel is less than the intensity of the white 
point of the image, the pixel is reclassified as smooth contone. 

22. (Original) The image processing method of claim 15, wherein the portion of 
the pixels comprises substantially all of the pixels of the image. 
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REMARKS 

Claims 1-22 are pending. By this Amendment, claims 1 and 8 are amended. 
Reconsideration in view of the foregoing amendments and the following remarks is 
respectfully requested. 

The courtesies extended to Applicants' representatives by Examiner Woldemariam at 
the interview held November 5, are appreciated. The reasons presented at the interview as 
warranting favorable action are incorporated into the remarks below and constitute 
Applicants* record of the interview. 

Claims 1 and 8 stand rejected under 35 U.S.C. §112, first paragraph as containing 
subject matter which was not described in the specification. Claims 1 and 8 are amended to 
obviate the rejection. Withdrawal of the rejection is respectfully requested. 

Claims 1-22 stand rejected under 35 U.S.C. §103 (a) over Lin (U.S. Patent Publication 
No. 2002/0076103 Al) in view of "Applicants' admitted prior art." The rejection is 
respectfully traversed. 

The Office Action recognizes that Lin does not disclose the ". . .background intensity 
level. . .based on substantially all of the pixels of the image. . feature of independent claims 
1 and 8, However, the Office Action asserts that (1) the disclosure in the background section 
that "[conventionally, background detection is performed by sampling pixel values either 
with a sub-region of the document (typically, the leading edge) or across the whole 
document" (emphasis added) correlates to the claimed feature, and (2) it would have been 
obvious to modify the Lin method and apparatus in a way that resulted in the combinations of 
features of claims 1 and 8. Applicants respectfully disagree. 
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The definition of " sampling " as used in Applicants 1 specification is commonly 
understood to refer to "a small portion, pieces, or segment." 1 This conventional method of 
determining intensity level of an image based on sampling pixel values is not. . ."a pixel 
classification. . .based on substantially all of the pixels of the image. . as recited in claims 1 
and 8. Thus, "Applicants 1 admitted prior art" would not have motivated one having ordinary 
skill in the art at the time of the invention to modify Lin to obtain the "substantially all of the 
pixels" feature recited in claims 1 and 8. 

Furthermore, Lin teaches away from the possibility of the claims 1 and 8 feature of 
"determining if reclassification is required. . ." since at the time Lin makes a first pass of the 
image Lin does not record the macro-detection or micro-detection results for each pixel of the 
image. See Paragraph [0060] of Lin. More specifically, because Lin does not record the 
macro-detection or micro-detection results for each pixel, it is impossible for each pixel to be 
reclassified, as recited in independent claims 1 and 8. 

Regarding independent claim 15, Lin fails to disclose "determining a background 
intensity level of an image, the background level being b ased on substantially all of the pixels 
of the image ", and "checking the classification of at least a portion of the pixels of the image 
based on the determined background intensity level of the image. ..." Lin classifies the 
intensity of each pixel based on the intensity of its surrounding pixels, not based on a 
background intensity level that is based on substantially all of the pixels of the image. See 
Lin Paragraph [0053]. Thus, one having ordinary skill in the art would not have modified Lin 
in view of Applicants 1 "admitted prior art" to obtain the combination of features recited in 
claim 15. 



1 American Heritage College Dictionary, fourth edition, page 1228. 
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Claims 2-7, 9-14 and 16-22 are patentable for at least the reasons mentioned above 
with respect to claims 1, 8 and 15. Withdrawal of the rejection is requested. 

ha view of the foregoing, it is respectfully submitted that this application is in 
condition for allowance. Favorable reconsideration and prompt allowance of claims are 
earnestly solicited. 

Should the Examiner believe that anything further would be desirable in order to place 
this application in even better condition for allowance, the Examiner is invited to contact the 
undersigned at the telephone number set forth below. 



Respectfully submitted, 



James A. Oliff 
Registration No. 27,075 



Robert G. Bachner 
Registration No. 60,122 



JAO:RGB/jls 



Date: November 1 2, 2008 



Oliff & Berridge, plc 
P.O. Box 320850 
Alexandria, Virginia 22320-4850 
Telephone: (703) 836-6400 



DEPOSIT ACCOUNT USE 

AUTHORIZATION 
Please grant any extension 

necessary for entry; 
Charge any fee due to our 
Deposit Account No. 24-0037 
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Art Unit: 2624 

DETAILED ACTION 
Response to Amendment 

1. Applicant's amendment field 11/12/2008 has been entered. Claims 1 and 8 
have been amended. Clams 1-22 are still pending, with claims, 1 , 8 and 15 being an 
independent. 

Response to Arguments 

2. Applicant's arguments field 1 1/12/2008 have been respectfully considered, but 
they are not persuasive. Examiner disagreed with applicant arguments, in claims 1 and 
8, claim limitations, "background intensity level based on substantially all of the pixels of 
the image and a pixel classification based on substantially all of the pixels of the image." 

Because Applicant's admitted prior art discloses background intensity level based on 
substantially all of the pixels of the image (conventionally background detection is 
performed by sampling pixel values either with a sub-region of the document or across 
the whole document (i.e., being based on substantially all of the pixel values (intensity 
level) of the image (see paragraph [0012]) and Lin discloses a pixel classification 
based on substantially all of the pixels of the image {see item 46, fig. 7 classification 
means and paragraph [0057] the statistics are examined in an attempt to classify each 
window. Windows that appear to contain primarily a single type of image data are 
classified according to their dominant image types and substantially referred to pixels 
size or pixel amount). Examiner disagreed with applicant's argument with claim 15 for 
above similar or identical reasons. Please see list of references by examiner. 
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Claim Rejections - 35 USC § 103 

3. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

4. Claims 1-22 are rejected under 35 U.S.C. 103(a) as being unpatentable over Lin 
et ah, "Lin" (U.S. Publication number 2002/0076103 A1 ) in view Applicant's admitted 
prior art" background section". 

Regarding claims 1 and 8, Lin discloses a pixel classification method and 
apparatus (see item 46, fig.7, classification means), comprising: 

determining a background intensity level of an image (see paragraph [0041], the 
output of the block based segmentation module 200 preferably is a three-layered mixed 
raster content file. Preferably, these layers represent background, foreground and 
[0047] and [0052] the image data comprises multiple scanlines of pixel image data, 
each scanline typically including intensity information for each pixel within the scanline. 
Typical image types include graphics, text, low-frequency halftone, high-frequency 
halftone, contone, etc). 

classifying a pixel of the image (see item 46, fig. 7 classification means and 
paragraph [0057] the statistics are examined in an attempt to classify each window. 
Windows that appear to contain primarily a single type of image data are classified 
according to their dominant image types); and 
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confirming the classification of the pixel based on the determined background intensity 
level of the image by comparing the intensity of the pixel with the determined 
background intensity level (see paragraph [0053], each pixel is examined and 
preliminary determination is made as to the image type of the pixel. In addition, the 
intensity of each pixel is compared to the intensity of its surrounding neighboring pixels. 
A judgment is made as to whether the intensity of the pixel under examination is 
significantly different than the intensity of the surrounding pixels); 

determining if reclassification is required; and reclassifying the pixel when 
reclassification is required (see paragraph [0054] and [0059] pixel is within a window 
that was classified as "mixed" during the first pass, micro-detection, macro-detection 
and windowing steps performed during the second pass are used to assign an image 
type to the pixel). 

Lin does not disclose the background intensity level being based on substantially 
all of the pixels of the image. However, Applicant's admitted prior art discloses 
conventionally background detection is performed by sampling pixel values either with a 
sub-region of the document or across the whole document (i.e., being based on 
substantially all of the pixel values (intensity level) of the image (see paragraph [0012]) 

It would have been obvious to ordinary skill in the art at the time when the 
invention was made to use Applicant's admitted prior art teachings to modify Lin's 
method by detecting the background based on substantially all of the pixels' intensity 
level of the image in order to more accurately reproduce the image, [Applicant's 
admitted prior art see paragraph [0009] lines 6-10]. 
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Regarding claim 2, Lin discloses the pixel classification method of claim 1, 
wherein the determining step comprises determining a white point of the image (see 
paragraph [0071] like wise, a common type of image is text of different colors on a white 
background), and Applicants admitted prior art further discloses determining at least 
one characteristic (background of substantially all of the pixel values of the image (see 
paragraph [0012] conventionally, background detection is performed by sampling pixel 
values either within a sub-region of document (typically, the leading edge) or across the 
whole document) and combined method of Lin and Applicant's admitted prior art 
determining a white point of the image based on at least one characteristic of 
substantially all of the pixels of the image). 

Regarding claim 3, Lin discloses the pixel classification method of claim 2, 
wherein the checking confirming step comprises comparing the intensity of the pixel 
with an intensity of the white point of the image (see paragraph [0052], [0053] and 
[0071] likewise, a common type image is text of different colors on a white background). 
Regarding claim 4, Lin further discloses the pixel classification method of claim 3, 
further comprising wherein the reclassifying step includes reclassifying the pixel as 
background when the pixel is classified as a class eligible to be reclassified and the 
intensity of the pixel is not less than the intensity of the white point of the image (see 
paragraph [0054], [0059] and [0071], likewise, a common type of image is text of 
different colors on a white back ground). 

Regarding claim 5, Lin discloses the pixel classification method of claim 3, further 
comprising wherein the reclassifying step includes reclassifying the pixel as one of 
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smooth contone and an equivalent class when the pixel is classified as background and 
the intensity of the pixel is less than the intensity of the white point of the image (see 
paragraph [0052], [0054], [0059] and [0071], likewise, a common type of image is text of 
different colors on a white back ground). 

Regarding claim 6, Lin discloses the pixel classification method of claim 1, 
wherein the identifies identifying a spread of intensity levels of the pixels of the image 
and determining step comprises determining an intensity level of a majority of the pixels 
(see paragraph [0053] and [0056] statistics are gathered and calculated for each of the 
window. The statistics are based on the intensity and macro-detection results for each 
of the pixels within a window), 

Applicants admitted prior art further discloses sampling pixel values either 
within a sub-region (as disclosed by Lin) or across the whole document image 
[paragraph [001 2] and the combined method of Lin and Applicant's admitted prior art is 
using the intensity level of substantially all pixels of the document. 

Regarding claim 7, Lin discloses the pixel classification method of claim 4, 
wherein the pixel is classified as smooth contone (see paragraph [0052] the image data 
comprises multiple typically including intensity information for each pixel within the 
scanline. Typical image types include graphics, text, low-frequency halftone, high- 
frequency halftone, contone, etc and paragraph [0057]). 

Regarding claim 9, Lin discloses the pixel classification apparatus of claim 8, 
wherein the background intensity level determining module determines a white point of 
the image (see paragraph [0041] the output of the block based segmentation module 
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200 preferably is a three-layered mixed raster content file. Preferably, these layers 
represent background, foreground and selectors fields and paragraph [0052] intensity 
and paragraph [0071] Likewise, a common type of image is text of different colors on a 
white background) and Applicant admitted prior art further discloses based on a 
characteristic of substantially all of the pixels of the image (see paragraph [0012] 
conventionally, background detection is performed by sampling pixel values either 
within a sub-region of document (typically, the leading edge) or across the whole 
document). 

Regarding claim 10, Lin discloses the pixel classification apparatus of claim 9, 
wherein the image processing module cheeks confirms the classification of the pixel by 
comparing the intensity of the pixel with the intensity of the white point of the image (see 
paragraph [0053], In addition, the intensity of each pixel is compared to the intensity of 
its surrounding neighboring pixels. A judgment is made as to whether the intensity of the 
pixel under examination is significantly different than the intensity of the surrounding 
pixels and paragraph [0071] likewise, a common type of image is text of different colors 
on a white background). 

Regarding claim 11, Lin discloses the pixel classification apparatus of claim 10, 
wherein when a pixel is classified as a class eligible to be reclassified and the intensity 
of the pixel is not less than the intensity of the white point of the image, the pixel is 
reclassified as background (see paragraph [0052], [0053], [0054] and [0071], Likewise, 
a common type of image is text of different colors on a white background). 



89 



Application/Control Number: 10/709,833 Page 8 

Art Unit: 2624 

Regarding claim 12, Lin discloses the pixel classification apparatus of claim 10, 
wherein when a pixel is classified as background and the intensity of the pixel is less 
than the intensity of the white point of the image, the pixel is reclassified as smooth 
contone (see paragraph [0052], [0053], [0054] and [0059] pixel is within a window that 
was classified as "mixed" during the first pass, the micro-detection, macro-detection and 
windowing steps performed during the second pass are used to assign an image type to 
pixel). 

Regarding claim 13 refer to claim 6 rejection. 
Regarding claim 14, Lin discloses the pixel classification apparatus of claim 1 1 , 
wherein the pixel is classified as one of smooth contone and an equivalent class (see 
paragraph [0053] and [0057] the image data comprises multiple scanlines of pixel image 
data, each scanline typically including intensity information for each pixel within the 
scanline. Typical image types include graphics, text, low-frequency halftone, high- 
frequency halftone, contone, etc). 

Regarding claim 15, Lin discloses an image processing method, comprising: 
determining a background intensity level of an image (see paragraph [0041], the output 
of the block based segmentation module 200 preferably is a three-layered mixed raster 
content file. Preferably, these layers represent background, foreground and [0047] and 
[0052] the image data comprises multiple scanlines of pixel image data, each scanline 
typically including intensity information for each pixel within the scanline. Typical image 
types include graphics, text, low-frequency halftone, high-frequency halftone, contone, 
etc); classifying a pixel of the image (see item 46, fig. 7 classification means and 
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paragraph [0057] the statistics are examined in an attempt to classify each window. 
Windows that appear to contain primarily a single type of image data are classified 
according to their dominant image types); and without adjusting an intensity of the pixel 
(see items 306 and 308, fig. 5„ reduced resolution is not reducing pixel intensity, 
reducing resolution by definition is reducing the number of pixel in image and not the 
pixel values); 

checking the classification of at least a portion of the pixels of the image based on 
the determined background intensity level of the image (see paragraph [0053], each 
pixel is examined and preliminary determination is made as to the image type of the 
pixel. In addition, the intensity of each pixel is compared to the intensity of its 
surrounding neighboring pixels. A judgment is made as to whether the intensity of the 
pixel under examination is significantly different than the intensity of the surrounding 
pixels); 

reclassifying pixels based on results of the checking step (see paragraph [0054] 
and [0059] if a pixel is within a window that was classified as "mixed" during the first 
pass, micro-detection, macro-detection and windowing steps performed during the 
second pass are used to assign an image type to the pixel); and processing image data 
of the pixels of the image based on the classification of the pixel (see paragraph [0063] 
the page segmentation and classification means 40 may also include image processing 
means 48 for processing the image data after each of the pixels has been labeled with 
an image type and as belonging to a particular window). 
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Lin does not disclose the background level being based on substantially all of the 
pixels of the image. However, Applicant's admitted prior art discloses conventionally 
background detection is performed by sampling pixel values either with a sub-region of 
the document or across the whole document (i.e., being based on substantially all of the 
pixel values (intensity level) of the image (see paragraph [0012]). 
It would have been obvious to ordinary skill in the art at the time when the invention 
was made to use Applicant's admitted prior art teachings to modify Lin's method by 
detecting the background based on substantially all of the pixels' intensity level of the 
image in order to more accurately reproduce the image, [Applicant's admitted prior art 
see paragraph [0009] lines 6-10]. 

Regarding claim 16, Lin discloses the image processing method of claim 15, 
further comprising storing a label associated with each of the pixels, wherein the label of 
each of the pixels is based on results of the classification step and the checking step for 
the pixel {see paragraph [0063] and [0064] classification means 40 may also include 
image processing means 48 for processing the image data after each of the pixels has 
been labeled with an image type and the image data obtaining means 36 could include 
a scanner or device for reading a stored image from a memory. The device might also 
include image data generation means 38 for generating image data to be segmented 
and classified by the two pass method)), 

Applicants admitted prior art further discloses sampling pixel values either within a 
sub-region (as disclosed by Lin) or across the whole document image {paragraph [0012] 
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and the combined method of Lin and Applicant's admitted prior art is using substantially 
all pixels of the document. 

Regarding claim 17, Lin discloses the image processing method of claim 15, 
wherein classifying a pixel of the image comprises classifying the pixel as one of 
smooth contone, rough contone, text, background, graphics and halftone (see 
paragraph [0052] and [0057] the image data comprises multiple scanlines of pixel image 
data, each scanline typically including intensity information for each pixel within the 
scanline. Typical image types include graphics, text, low-frequency halftone, high- 
frequency halftone, contone, etc). 

Regarding claim 18, refer to claim 2 rejection. 
Regarding claim 19, Lin discloses the image processing method of claim 18, 
wherein the checking step comprises comparing an intensity of the pixel with an 
intensity of the white point of the image (see paragraph [0053] each pixel is examined 
and preliminary determination is made as to the image type of the pixel In addition, the 
intensity of its surrounding neighboring pixels. A judgment is made as to whether the 
intensity of the pixel under examination is significantly different than the intensity of the 
surrounding pixels and [0071], likewise, a common type of image is text of different 
colors on a white background). 

Regarding claim 20, Lin discloses the image processing method of claim 19, 
wherein when the pixel is classified as smooth contone and the intensity of the pixel is 
not less than the intensity of the white point of the image, the pixel is reclassified as 
background (see paragraph [0052], [054] and [0057] the statistics are examined in an 
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attempt to classify each window. Windows that appear to contain primarily a single type 
of image data are classified according to their dominant image types. Windows that 
contain more than one type of image are classified as "mixed"). 

Regarding claim 21, Lin discloses the image processing method of claim 19, 
wherein when the pixel is classified as background and the intensity of the pixel is less 
than the intensity of the white point of the image, the pixel is reclassified as smooth 
contone (see paragraph [0052], [054] and [0057] the statistics are examined in an 
attempt to classify each window. Windows that appear to contain primarily a single type 
of image data are classified according to their dominant image types. Windows that 
contain more than one type of image are classified as "mixed"). 

Regarding claim 22, Applicant's admitted prior art further discloses the image 
processing method of claim 15, wherein the portion of the pixels comprises substantially 
all of the pixels of the image (see paragraph [0012] conventionally, background 
detection is performed by sampling pixel values either within a sub-region of document 
(typically, the leading edge) or across the whole document). 

Conclusion 

5. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to AKLILU k. WOLDEMARIAM whose telephone number is 
(571)270-3247. The examiner can normally be reached on Monday-Thursday 6:30 a.m- 
5:00 p.m EST. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Samir Ahmed can be reached on 571-272-7413. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Samir Ahmed 
Examiner 
Art Unit 2624 

/A. k. W./ 

Examiner, Art Unit 2624 
02/01/2009 

/Brian Q Le/ 

Primary Examiner, Art Unit 2624 
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In reply to the February 10, 2009 Office Action, reconsideration of the rejection is 
respectfully requested in light of the following remarks. 



Sir: 
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REMARKS 

Claims 1-22 are pending. Reconsideration in view of the following remarks is 
respectfully requested. 

I. All Pending Claims are Patentable 

Claims 1-22 stand rejected under 35 U.S.C. §103(a) over Lin (U.S. Patent Publication 

No. 2002/0076103) in view of Applicants* "admitted prior art background section". The 
rejection is respectfully traversed. 

In responding to Applicants 1 arguments filed on November 12, 2008, the Office 
Action alleges that Applicants 1 Background section discloses determining the background 
intensity level based on substantially all of the pixels of the image and that Lin discloses a 
pixel classification based on substantially all the pixels of the image. Applicants respectfully 
disagree. 

Applicants 1 Description of Related Art section discloses that "background detection is 
performed by sampling pixel values either within a sub-region of the document (typically, 
leading end) or across the whole documents" (emphasis added). The use of " sampling " in 
Applicants' specification is commonly understood to refer to "a small portion, pieces, or 
segment." 1 This conventional method of determining intensity level of an image based on 
sampling pixel values is not. . ."a pixel classification. . .based on substantially all the pixels of 
the image. . ." as recited in independent claims 1 and 8. Applicants' specification is consistent 
with this ordinary meaning of "sampling" in that Applicants' specification explicitly indicates 
that determining "based on substantially all the pixels. . ." is different from "sampling". See, 
for example, paragraphs [0012], [0014] and [0058] of the specification. Thus, "Applicants' 
admitted prior art" does not disclose that it was known to determine background intensity 

1 American Heritage College Dictionary, fourth edition, pages 1228 
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level based on " substantially all of the pixels " as recited in independent claims 1 and 8, and 
furthermore would not have motivated one having ordinary skill in the art at the time of the 
invention to modify Lin to obtain the combination of features recited in claims 1 and 8. 

Additionally, Office Action does not respond to Applicants' argument tha t Lin teaches 
awav from the claims 1 and 8 feature of "determining if reclassification is required . . . " since 
at the time Lin makes a first pass of the image Lin does not record the macro-detection or 
micro-detection results from the pixel of the image. See Lin paragraph [0060]. Specifically, 
because Lin does not record the macro-detection or micro-detection results of the pixel, it is 
impossible for each pixel to be reclassified, as recited in independent claims 1 and 8. 

Regarding independent claim 15, Lin and "admitted prior art" fail to disclose or 
render obvious "determining a background intensity level of an image, the background level 
being based on substantially all of the pixels of the image ", and "checking the classification of 
at least a portion of pixels of the image based on the determined background intensity level of 
the image. . ." Lin classifies the intensity of each pixel based on the intensity of its 
surrounding pixels, not based on a background intensity level that is based on substantially all 
of the pixels of the image. See Lin paragraph [0053]. Thus, one having ordinary skill in the 
art would not have modified Lin in view of Applicants' "admitted prior art" to obtain the 
combination of features recited in claim 15. 

Thus, claims 1, 8 and 15 are patentable. Accordingly, claims 2-7, 9-14 and 16-22 are 
also patentable for at least the reasons explained above with respect to claims 1, 8 and 15. 
Withdrawal of the rejection is requested. 

II. Conclusion 

In view of the foregoing, it is respectfully submitted that this application is in 

condition for allowance. Favorable reconsideration and prompt allowance of all pending 
claim are earnestly solicited. 
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Should the Examiner believe that anything further would be desirable in order to place 
this application in even better condition for allowance, the Examiner is invited to contact the 



undersigned at the telephone number set forth below, 



Respectfully submitted, 



James A. Oliff 
Registration No. 27,075 

Robert G. Bachner 
Registration No. 60,122 



JAO:RGB/jls 

Date: March 10, 2009 

Oliff & Berridge, flc 

P.O. Box 320850 

Alexandria, Virginia 22320-4850 

Telephone: (703) 836-6400 



DEPOSIT ACCOUNT USE 

AUTHORIZATION 
Please grant any extension 

necessary for entry; 
Charge any fee due to our 
Deposit Account No. 24-0037 
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Applicant argued that about references Lin (U.S. Patent Publication No. 2002/0076103 and Applicants' "admitted prior art background 
section", for example claim limitations, " background intensity level based on substantially all of the pixels of the image and a pixel 
classification based on substantially all of the pixels of the image", Examiner disgreed with applicant's argument, 

bcause Applicant's admitted prior art discloses background intensity level based on substantially all of the pixels of the image 
(conventionally background detection is performed by sampling pixel values either with a sub-region of the document or across the whole 
document (i.e., being based on substantially all of the pixel values (intensity level) of the image (see paragraph [0012]) and Lin discloses a 
pixel classification based on substantially all of the pixels of the image (see item 46, fig. 7 classification means and paragraph [0057] the 
statistics are examined in an attempt to classify each window. Windows that appear to contain primarily a single type of image data are 
classified according to their dominant image types and substantially referred to pixels size or pixel amount). 
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REMARKS 

Claims 1-22 are pending. Review of the February 10, 2009 Final Rejection in view of 
the following remarks is respectfully requested. 

Claims 1-22 stand rejected under 35 U.S.C. §103(a) over Lin (U.S. Patent Publication 
No. 2002/0076103) in view of Applicants' "admitted prior art background section". The 
rejection is improper and should be withdrawn. 

The Final Rejection commits clear error when it asserts that Applicants' "admitted 
prior art background section" discloses that it was known to determine a background intensity 
level of an image "based on substantially all of the pixels of the image." Because the above- 
quoted feature is present in each of Applicants' independent claims (claims 1, 8 and 15), 
reversing the Examiner's decision should result in withdrawal of the rejection, which is the 
only rei ection, and allowance of this application. 

In responding to Applicants' arguments filed on November 12, 2008, the Office 
Action alleges that Applicants' Background section discloses that it was known to determine 
the background intensity level based on substantially all of the pixels of the image and that 
Lin discloses a pixel classification based on substantially all the pixels of the image. 
Applicants respectfully disagree. 

Applicants' Description of Related Art section discloses that "background detection is 
performed by sampling pixel values either within a sub-region of the document (typically, 
leading end) or across the whole documents" (emphasis added). In particular, paragraph 
[0012] states: 

Conventionally, background detection is performed by sampling pixel values 
either within a sub-region of the document (typically, the leading edge) or 
across the whole document. For conventional processes, only a portion (i.e., 
not the full document) is used to detect the background of the document to be 
reproduced. The detected lead-edge or other sub-region background 
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information is then used to process and classify each of the pixels of the 
scanned image. 

The use of " sampling " in Applicants' specification is commonly understood to refer to "a 

small portion, pieces, or segment." 1 This conventional method of determining intensity level 

of an image based on sampling pixel values is not. , ."a pixel classification. . .based on 

substantially all the pixels of the image. . ." as recited in independent claims 1 and 8. 

"Sampling" is the antithesis of "substantially all." Applicants' specification is consistent with 

this ordinary meaning of "sampling" in that Applicants 1 specification explicitly indicates that 

determining " based on substantially all the pixels . . ." is different from "sampling". See, for 

example, paragraphs [0014] and [0056] of Applicants' specification (especially last sentence 

of paragraph [0056]), which are reproduced below. 

Various exemplary embodiments of the invention provide a pixel 
classification method for classifying pixels of an image by determining a 
background intensity level of an image which is based on substantially all of 
the pixels of the image. The method also involves checking the 
classification of the pixel based on the determined background intensity 
level of the image. 

Various exemplary embodiments of the invention may be incorporated into 
the exemplary segmentation and processing method described above. In 
particular, various exemplary embodiments of the invention use the results 
of a full page based background detection to adjust, as necessary, the 
classification of the pixels by checking the classification. Various 
exemplary embodiments of the invention check the classification of a pixel 
by comparing the intensity of the pixel with the intensity of the white point 
or the background intensity level of the image. The white point or the 
background intensity level of the image is determined based on an analysis 
of substantially all of the pixels of the document, and not just a sampling of 
the pixels or a sub-region of the image , (emphasis added) 

Thus, "Applicants' admitted prior art" does not disclose that it was known to determine 
background intensity level based on " substantially all of the pixels " as recited in independent 
claims 1 and 8, and furthermore would not have rendered it obvious to one having ordinary 
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skill in the art at the time of the invention to modify Lin to obtain the combinations of 
features recited in claims 1 and 8. 

Additionally, the Office Action does not respond to Applicants' argument tha t Lin 
teaches away from the claims 1 and 8 feature of "determining if reclassification is required . . 
because at the time Lin makes a first pass of the image, Lin does not record the macro- 
detection or micro-detection results from the pixel of the image. See Lin paragraph [0060], 
which states: 

Once each portion of the image data has been classified according to 
standard image types, further processing of the image data can be efficiently 
performed. Because the micro-detection and macro-detection results from 
the first pass are not recorded for each pixel of the image, the memory 
requirements for a device embodying the invention are minimized. This 
helps to minimize the cost of such an apparatus. 

Applicants* specification explicitly acknowledges systems such as Lin's and distinguishes the 
features of the claims, for example, at paragraph [0013], which states: 

In known two-pass methods, for example, the original classification of a 
pixel as background is done during the first pass using lead-edge or other 
sub-region information and pixels classified as background during the first 
pass are not re-classified during the second pass. As lead-edge or other sub- 
region information may not be a true indication of the background of the 
captured image, rnisclassification of pixels as background can occur. For 
example, a background pixel can be classified as smooth contone or vise 
versa. Similarly, in known two-pass methods, pixels are subjected to a 
second pass when the pixel was associated with a "mixed" window during 
the first pass. Thus, in known classification methods, the classification of a 
pixel is not reconsidered. However, as discussed above, because it may be 
advantageous to classify pixels of different image types differently, the 
rnisclassification of a pixel as background, for example, can affect 
background suppression and also the rendering of the types of pixels. 

Lin performs the pixel classification as explained in Applicants 1 Background of the Invention 

section. Thus, because Lin does not record the macro-detection or micro-detection results of 



1 American Heritage College Dictionary, fourth edition, page 1228 
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the pixel, it is im possible fo r each pixel to be reclassified, as recited in independent claims 1 
and 8. 

Regarding independent claim 15, Lin and "admitted prior art" fail to disclose or 
render obvious "determining a background intensity level of an image, the background level 
being based on substantially all of the pixels of the image ", and "checking the classification of 
at least a portion of pixels of the image based on the determined background intensity level of 
the image. . ." Lin classifies the intensity of each pixel based on the intensity of its 
surrounding pixels, not based on a background intensity level that is based on substantially all 
of the pixels of the image. See Lin paragraph [0053]. As explained above, the "admitted 
prior art" also does not disclose this feature. Thus, one having ordinary skill in the art would 
not have modified Lin in view of Applicants' "admitted prior art" to obtain the combination of 
features recited in claim 15. 

Thus, claims 1, 8 and 15 are patentable. Accordingly, claims 2-7, 9-14 and 16-22 are 
also patentable for at least the reasons explained above with respect to claims 1 , 8 and 15. 
Withdrawal of the rejection is requested. 
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(57) ABSTRACT 

This invention relates to a method and apparatus for seg- 
menting an image using a combination of image segmenta- 
tion techniques. More particularly, the invention is directed 
to an improved image segmentation technique for use in an 
image processing system that performs at least two distinct 
image segmentation processes on an image and combines 
the results to obtain a combined multi-layer representation 
of the image that can be suitably processed. In a specific 
example, a block based segmentation technique is per- 
formed on an image to generate a MRC (mixed raster 
content) representation — having foreground, background 
and selector layers. A pixel based segmentation technique is 
also performed on the image to generate rendering hints. The 
MRC representation and the rendering hints are then com- 
bined to obtain a four (4) layer representation of the image. 
The four layer representation is subsequently processed as 
required by the image processing system, e.g. compressed 
and stored. 
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METHOD AND APPARATUS FOR SEGMENTING 
AN IMAGE USING A COMBINATION OF IMAGE 
SEGMENTATION TECHNIQUES 

BACKGROUND OF THE INVENTION 

[0001] This invention relates to a method and apparatus 
for segmenting an image using a combination of image 
segmentation techniques. More particularly, the invention is 
directed to an improved image segmentation technique for 
use in an image processing system that performs at least two 
distinct image segmentation processes on an image and 
combines the results to obtain a combined multi-layer rep- 
resentation of the image that can be suitably processed. In a 
specific example, a block based segmentation technique is 
performed on an image to generate a MRC (mixed raster 
content) representation — having foreground, background 
and selector layers. A pixel based segmentation technique is 
also performed on the image to generate rendering hints. The 
MRC representation and the rendering hints are then com- 
bined to obtain a four (4) layer representation of the image. 
The four layer representation is subsequently processed as 
required by the image processing system, e.g. compressed 
and stored. 

[0002] While the invention is particularly directed to the 
art of combining image segmentation techniques to obtain a 
useful result, and will be thus described with specific ref- 
erence thereto, it will be appreciated that the invention may 
have usefulness in other fields and applications. 

[0003] By way of background, various methods for seg- 
menting images are known. In general, such image segmen- 
tation methods are implemented to satisfy a wide variety of 
image processing needs. For example, when an image is 
sought to be compressed, it is advantageous to first deter- 
mine the types of objects (e.g. continuous tone objects, 
background portions, text, , . . etc.) that are contained in the 
image. Compression techniques, depending on their precise 
nature, tend to most effectively compress only certain types 
of image objects. Thus, images are segmented by object type 
so that appropriate compression techniques may be applied 
to each of the respective object types of the image. To 
illustrate, it is well known in the image processing field that 
JPEG compression techniques work fairly well on continu- 
ous tone pixel maps but do not operate effectively on text. 
Conversely, the Lempel-Ziv Welch compression techniques 
do not perform adequately on scanned pixel maps. 

[0004] Moreover, the various types of image segmentation 
methods each possess relative strengths. For example, pixel 
based image segmentation methods allow for improved 
image rendering capabilities over other segmentation meth- 
ods. In this regard, pixel level segmentation methods gen- 
erate pixel level rendering hints — which are pieces of infor- 
mation that indicate certain characteristics about an image. 
For example, a rendering hint may indicate the location of 
an edge within an image. Corresponding windows (whereby 
all pixels within a window have the same rendering hints) 
are also utilized. Although the generation of rendering hints 
and categorization using window identifications are advan- 
tageous features of pixel level segmentation from the stand- 
point of image rendering, a severe disadvantage of such 
methods is that compression ratios of a pixel based seg- 
mented image are not acceptable for many applications. 

[0005] Other image segmentation methods that are well 
known are referred to as block based segmentation methods. 



That is, the subject image is segmented on a block-by-block 
basis as opposed to a pixel-by-pixel basis. Block based 
image segmentation methods attain improved compression 
ratios over pixel based methods and also are conducive to 
generating mixed raster content (MRC) data for ease of 
compression. The disadvantage of block based image seg- 
mentation methods, however, is that rendering hints are not 
effectively generated using these methods. Even if they are 
generated, use thereof tends to place artifacts on the ren- 
dered image. 

[0006] As such, a segmentation system that combines the 
advantages of the above referenced segmentation methods, 
and others, and utilizes such advantages for improved ren- 
dering is desired. 

[0007] The present invention contemplates a new and 
improved image segmentation method and apparatus that 
resolves the above-referenced difficulties and others. 

SUMMARY OF THE INVENTION 

[0008] A method and apparatus for segmenting an image 
using a combination of image segmentation techniques are 
provided. 

[0009] In one aspect of the invention, a method comprises 
steps of obtaining image data, inputting the image data into 
a first image segmentation module, generating first segmen- 
tation data by the first image segmentation module, the first 
image segmentation data representing at least one first 
characteristic of the image data, inputting the image data 
into a second image segmentation module, generating sec- 
ond image segmentation data by the second image segmen- 
tation module, the second image segmentation data repre- 
senting at least one second characteristic of the image data, 
and integrating the first image segmentation data with the 
second image segmentation data to obtain modified image 
data. 

[0010] In another aspect of the invention, inputting the 
image data to the first image segmentation module and 
inputting of the image data to the second image segmenta- 
tion module are accomplished concurrently. 

[0011] In another aspect of the invention, the generating of 
the first image segmentation data comprises generating first 
characteristic data representing a background layer, a selec- 
tor layer, and a foreground layer of the image data. 

[0012] In another aspect of the invention, the generating of 
the second image segmentation data comprises generating 
second characteristic data representing rendering hints, 

[0013] In another aspect of the invention, the system for 
implementing the method according to the present invention 
is provided. 

[0014] In another aspect of the invention, the system 
comprises means for obtaining image data, means for gen- 
erating first segmentation data, the first image segmentation 
data representing at least one first characteristic of the image 
data, means for generating second image segmentation data, 
the second image segmentation data representing at least one 
second characteristic of the image data and means for 
integrating the first image segmentation data with the second 
image segmentation data to obtain modified image data. 

[0015] In another aspect of the invention, an image ren- 
dering system adapted for segmenting an image comprises a 
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scanner operative to obtain image data, a bitmap generator 
operative to generate a bitmap corresponding to the image 
data, a first image segmentation module operative to gen- 
erate first image segmentation data, the first image segmen- 
tation data representing at least one first characteristic of the 
image data, a second image segmentation module operative 
to generate second image segmentation data, the second 
image segmentation data representing at least one second 
characteristic of the image data, a combining module opera- 
tive to combine the first image segmentation data with the 
second image segmentation data to obtain modified image 
data, a compression module operative to compress the 
modified image data, a storage module operative to store the 
compressed image data, a decompression module operative 
to decompress the stored image data and a print engine 
operative to render the image based on the decompressed 
data. 

[0016] In another aspect of the invention, the system is 
included in a xerographic printing environment. 

[0017] In another aspect of the invention, the print engine 
is a xerographic print engine. 

[0018] In another aspect of the invention, the first image 
segmentation data comprises first characteristic data repre- 
senting a background layer, a selector layer and a foreground 
layer of the image data. 

[0019] In another aspect of the invention, the second 
image segmentation data comprises second characteristic 
data representing rendering hints. 

[0020] Further scope of the applicability of the present 
invention will become apparent from the detailed descrip- 
tion provided below. It should be understood, however, that 
the detailed description and specific examples, while indi- 
cating preferred embodiments of the invention, are given by 
way of illustration only, since various changes and modifi- 
cations within the spirit and scope of the invention will 
become apparent to those skilled in the art. 

DESCRIPTION OF THE DRAWINGS 

[0021] The present invention exists in the construction, 
arrangement, and combination of the various parts of the 
device, and steps of the method, whereby the objects con- 
templated are attained as hereinafter more fully set forth, 
specifically pointed out in the claims, and illustrated in the 
accompanying drawings in which: 

[0022] FIG. 1 is a block diagram of an image-rendering 
device incorporating the present invention; 

[0023] FIG. 2 is a block diagram of an alternative image- 
rendering device incorporating the present invention; 

[0024] FIG. 3 is a block diagram of an image segmenta- 
tion module according to the present invention; 

[0025] FIG. 4 is a block diagram illustrating a more 
specific embodiment of an image segmentation module 
according to the present invention; 

[0026] FIG. 5 is a flowchart illustrating an exemplary 
block based segmentation scheme; 

[0027] FIG. 6 is a flowchart illustrating an exemplary 
pixel based segmentation scheme; 



[0028] FIG. 7 is a block diagram of a system for imple- 
menting the scheme of FIG. 6; and, 

[0029] FIG. 8 is a flow chart illustrating the method 
according to the present invention. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS 

[0030] The present invention is directed to an image 
segmentation method and apparatus that combines the out- 
put of a variety of image segmentation modules to obtain an 
integrated result. This technique allows for the advantageous 
use of various segmentation methods on the same image. In 
this way, the same image is segmented a variety of different 
ways so that, depending on the ultimate use of the output 
obtained as a result of the implementation of the present 
invention, advantages of these various methods may be 
realized. 

[0031] Referring now to the drawings wherein the show- 
ings are for purposes of illustrating the preferred embodi- 
ments of the invention only and not for purposes of limiting 
same, FIG. 1 provides a view of an image rendering system 
into which the present invention is preferably incorporated. 
As shown, an image rendering system 10 includes a scanner 
12 that scans an input document 14. The data output from the 
scanner is then input to a pixel map generator 16, which 
generates a pixel map that is stored in buffer storage 18. The 
data stored in the buffer storage 18 is then input to an image 
processing unit 20. The image processing unit 20 includes a 
segmentation module 22, a compression module 24, and a 
storage module 26. The image processing unit 20 also 
includes a processing and control module 28 as well as a 
decompression module 30. The image processing unit 20 is 
connected at its output to a print engine 32 which is 
operative to render an image in an output document 34. 

[0032] It should be appreciated that the scanner 12, pixel 
map generator 16, and the storage buffer 18 are of conven- 
tional design and are operative to generate a scanned, 
rasterized image and a corresponding pixel map therefore, so 
that the image may be digitally stored in the buffer. This is 
accomplished using well-known techniques in the image 
rendering and image processing fields. 

[0033] The image processing unit 20 performs conven- 
tional image processing techniques on the image in addition 
to performing steps of the method according to the present 
invention. As such, the image processing unit 20 is generally 
controlled by control module 28, but may well contain 
components that are not specifically shown in FIG. 1. In 
addition, the configuration and function of the compression 
module 24 and decompression module 30 depend on the 
types of compression and decompression schemes that are 
used in conjunction with the present invention. Many suit- 
able compression and decompression schemes are well- 
known to those of skill in this field. Further, the storage 
device 26 may take any suitable form that can accommodate 
storage of compressed image data. 

[0034] It is also to be understood that the print engine 32 
is preferably a xerographic print engine that is well known 
in the art and can be used in a xerographic printing envi- 
ronment. However, any suitable print engine will suffice. 

[0035] Referring now to FIG. 2, an alternative environ- 
ment into which the present invention may be incorporated 
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is illustrated. More specifically, an image rendering system 
50 includes an image processing unit 52 that is operative to 
receive a pixel map 54, process the pixel map 54, and output 
appropriate data based on the processing to a network 56 or 
print engine 58. 

[0036] It is to be appreciated that the pixel map 54 may be 
generated within a network and digitally stored prior to its 
input to the image processing unit 52. Other various known 
ways of generating a pixel map or a bit map may also be 
used. The image processing unit 52 takes the form similar to 
the image processing unit 20 in that it includes the segmen- 
tation module 60, the processing and control module 62, a 
compression module 64, a storage module 66 and a decom- 
pression module 68. Of course, if the compressed image is 
transmitted over a network 56, the decompression of that 
image will preferably occur at the receiver. For example, 
decompression may occur within a fax machine, another 
processing terminal, . . . etc. The network 56 may be a local 
area network, a wide area network, or the Internet. Like the 
print engine 32 of FIG. 1, it is also to be understood that the 
print engine 58 is preferably a xerographic print engine that 
is well known in the art and can be used in a xerographic 
printing environment. However, any suitable print engine 
will suffice. 

[0037] As noted above, the present invention may be 
applied in both of the environments described in connection 
with FIGS. 1 and 2, as well as others. In either case shown, 
referring now to FIG. 3, the segmentation module 22 (or 60) 
includes a plurality of distinct segmentation modules 100. 
Each such segmentation module 100 is operative to generate 
image segmentation data representing various characteris- 
tics of the image data. Of course, it is to be appreciated that 
segmentation becomes more robust with the addition of 
farther segmentation modules. 

[0038] This image segmentation data is then combined, or 
integrated, at module 102 and the overall segmentation 
result is output. Preferably, the segmentation modules 100 
are each distinct from one another to the extent that they 
generate image segmentation data representing different 
characteristics of the image data. The combining module 
102 then preferably generates data representing all of the 
characteristics for which segmentation was accomplished by 
integrating the results of each of the segmentation modules. 
This data can then be used advantageously for a variety of 
desired purposes within any image rendering or networked 
computer system, as illustratively shown in FIGS. 1 and 2. 

[0039] With reference to FIG. 4, a more specific embodi- 
ment according to the present invention is illustrated. It is to 
be appreciated that the segmentation module illustrated in 
FIG. 4 is but one preferred embodiment of the present 
invention. It will be further appreciated that the details of 
implementation may be modified to suit the needs of a 
particular application. For example, different segmentation 
techniques than are shown could be used. 

[0040] As shown, however, the image segmentation mod- 
ule 22 (or 60) is provided with a block based (or, more 
generally, object based) image segmentation module 200 
and a pixel based image segmentation module 202, The 
input image is provided to each of these segmentation 
modules. 

[0041] The output of the block based segmentation mod- 
ule 200 preferably is a three-layered mixed raster content 



file. Preferably, these layers represent background, fore- 
ground, and selector fields. Likewise, the pixel based image 
segmentation module 202 has an output. This output 
includes rendering hints. The outputs of each of the image 
segmentation modules 200 and 202 are combined at com- 
bining module 204 to produce a four layered mixed raster 
content (MRC) data file. Again, this modified (e.g. opti- 
mized) image data is utilized in manners accommodated by 
the particular system into which the invention is incorpo- 
rated. Preferably, the data in the form using MRC represen- 
tation is in a device independent color space such as Lab. As 
to the environment shown in FIGS. 1 and 2, the data is 
compressed, stored, transmitted and/or used in rendering. 

[0042] With respect to the operation of the block based 
image segmentation module 200, any blocked or object 
based technique that outputs a layered MRC representation 
will suffice. However, a preferred exemplary segmentation 
technique that produces a three layered output is illustrated 
in U.S. Pat. No. 5,778,092 to MacLeod et al. (issued Jul. 7, 
1998) entitled "Method and Apparatus For Compressing 
Color or Gray Scale Documents," commonly assigned and 
incorporated herein by reference. 

[0043] As disclosed therein, the pixel map representing a 
color or gray-scale document is decomposed into a three- 
plane page format. The document format is comprised of a 
"foreground" plane, a "background" plane, and a "selector" 
plane. The "foreground" and "background" planes are stored 
at the same bit depth and number of colors as the original 
raw pixel map, but usually at reduced resolution. The 
"selector" plane is created and stored as a 1 bit per pixel 
bitmap. 

[0044] Each of the planes is comprised of a corresponding 
map that may be ultimately compressed using a compression 
method suitable for the data contained in the plane, as 
referenced above. For example, the foreground and back- 
ground planes may be compressed and stored using JPEG, 
while the selector plane may be compressed and stored using 
a symbol-based compression format. It would be apparent to 
one of skill in the art to compress and store the planes using 
other formats that are suitable for the intended use of the 
color document. 

[0045] The "background" plane contains two things. First, 
it contains the color of the "background" of the page, 
including the color of tints, washes, etc. Second, it contains 
the continuous-tone pictures that are found on the page. The 
"foreground" or "ink" plane contains the "ink colors" of 
foreground items such as text. 

[0046] The "selector" plane is stored at a higher resolution 
(e.g. 600 spots/inch for a 300 spots/inch original document). 
The purpose of the selector plane is to describe, for each 
pixel in the selector plane, whether to use the pixel value 
found in the background plane or the foreground plane. A 
"white" pixel in the selector plane (i.e. a logical zero value) 
means the pixel value should be taken from the correspond- 
ing pixel from the background plane. A "black" pixel in the 
selector plane (i.e. a logical one value) means that the pixel 
value should be taken from the corresponding pixel from the 
foreground plane. Preferably, the selector, foreground and 
background planes are brought to the same resolution before 
the selector is used to generate the single plane image. 

[0047] Referring to FIG. 5, a pixel map representation of 
the document is obtained (step 302). This may be through 
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scanning an original of the document, or by retrieving a 
stored pixel map representation of the document, depending 
on whether the system of FIG. 1 or FIG. 2 is implemented, 
for example. The pixel map representation is then analyzed 
to generate the information for the three planes. The first step 
of the analysis is to create a thresholded bitmap from the 
original pixel map (step 304). Generally, the thresholding 
step creates a new bitmap containing binary values is formed 
based on an adaptively determined threshold value. The 
thresholded bitmap is desirable since it eliminates undesir- 
able artifacts. Using the threshold bitmap, a reduced reso- 
lution foreground map containing color (or gray scale) 
information of foreground items, such a text is computed 
(step 306). 

[0048] The reduced resolution background map is then 
computed (step 308). In this step, the "image" or non-text 
portions are identified by an image segmentation process. 
This information is used to create the reduced resolution 
background map which contains background color informa- 
tion as well as continuous tone image information. 

[0049] Next, the selector plane is computed (step 310). 
The selector plane is a bitmap computed using a suitable 
thresholding technique on the original pixel map. Of course, 
optionally, the layers are then compressed if required by the 
system (step 312). 

[0050] With respect to the pixel based image segmentation 
module 202, any of a variety of pixel based schemes the are 
well known in the field may be utilized. Preferably, however, 
the pixel based scheme described in U.S. Pat. No, 5,850,474 
to Fan et al (issued Dec. 15, 1998) entitled "Apparatus and 
Method For Segmenting and Classifying Image Data" and 
U.S. Pat. No. 5,293,430 to Shiau et al. (issued Mar. 8, 1994) 
entitled "Automatic Image Segmentation Using Local Area 
Maximum and Minimum Image Signals", both of which are 
commonly assigned and incorporated herein by reference, is 
used. 

[0051] In the disclosed method, as recited in U.S. Pat. No. 
5,850,474, a block diagram of a two pass segmentation and 
classification method embodying the invention is shown in 
FIG. 6. The method segments a page of image data into 
windows, classifies the image data within each window as a 
particular image type and records information regarding the 
window and image type of each pixel. Once the image type 
for each window is known, further processing of the image 
data can be efficiently performed. 

[0052] The image data comprises multiple scanlines of 
pixel image data, each scanline typically including intensity 
information for each pixel within the scanline. Typical 
image types include graphics, text, low-frequency halftone, 
high-frequency halftone, contone, etc. 

[0053] During a first step S101, micro-detection is carried 
out. During micro-detection, multiple scanlines of image 
data are buffered into memory. Each pixel is examined and 
a preliminary determination is made as to the image type of 
the pixel. In addition, the intensity of each pixel is compared 
to the intensity of its surrounding neighboring pixels. A 
judgment is made as to whether the intensity of the pixel 
under examination is significantly different than the intensity 
of the surrounding pixels. When a pixel has a significantly 
different intensity than it neighboring pixels, the pixel is 
classified as an edge pixel. 



[0054] During a second step S103, a macro-detection is 
performed. During the macro-detection step, the results of 
the micro-detection step are used to identify those pixels 
within each scanline that are edges and those pixels that 
belong to image runs, The image type of each image run is 
then determined based on the micro-detection results. The 
image type of an image run may also be based on the image 
type and a confidence factor of an adjacent image run of a 
previous scanline. Also, if an image run of a previous 
scanline was impossible to classify as a standard image type, 
but information generated during examination of the present 
scanline makes it possible to determine the image type of the 
image run of the previous scanline, that determination is 
made and the image type of the image run of the previous 
scanline is recorded. 

[0055] In the next step S105, the image runs of adjacent 
scanlines are combined to form windows. A window is thus 
a contiguous area, of an arbitrary shape, in an image where 
all pixels are of the same class. 

[0056] In the next step S107, statistics are gathered and 
calculated for each of the windows. The statistics are based 
on the intensity and macro-detection results for each of the 
pixels within a window. 

[0057] In the next step S109, the statistics are examined in 
an attempt to classify each window. Windows that appear to 
contain primarily a single type of image data are classified 
according to their dominant image types. Windows that 
contain more than one type of image are classified as 
"mixed". 

[0058] At the end of the first pass, in step S110, the 
beginning point and the image type of each of the windows 
is recorded. 

[0059] During the second pass, in steps Sill, S113 and 
SH5, the micro-detection, macro-detection and window 
generation steps, respectively, are repeated. In the next step 
S117, labeling of the pixels occurs. During the labeling step, 
information about the image type and the window of each 
pixel is recorded. If a pixel is within a window that was 
classified as "mixed" during the first pass, the micro-detec- 
tion, macro-detection and windowing steps performed dur- 
ing the second pass are used to assign an image type to the 
pixel. At the end of the labeling step, each pixel is labeled 
as a particular image type. 

[0060] Once each portion of the image data has been 
classified according to standard image types, further pro- 
cessing of the image data can be efficiently performed. 
Because the micro-detection and macro-detection results 
from the first pass are not recorded for each pixel of the 
image, the memory requirements for a device embodying 
the invention are minimized. This helps to minimize the cost 
of such an apparatus. 

[0061] A block diagram of a page segmentation and clas- 
sification apparatus capable of performing the two pass 
method is shown in FIG. 7. The page segmentation and 
classification means 40 includes micro-detection means 42 
for performing the micro-detection step, macro-detection 
means 43 for performing the macro-detection step and 
windowing means 44 for grouping the image runs of the 
scanlines together to form windows. The apparatus also 
includes statistics means 45 for gathering and calculating 
statistics regarding the pixels within each window and 
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classification means 46 for classifying each of the windows 
as a particular image type based on the gathered statistics. 

[0062] Memory means 47 are provided for recording the 
beginning points and image types of each of the windows 
and the beginning points and image types of any initially 
unknown image runs that were subsequently classified dur- 
ing the first pass. The memory means 47 may also be used 
to store the window and image type of each pixel at the end 
of the second pass. Typically, however, the image data is 
used immediately to process, transmit and/or print the 
image, and the image data is discarded. 

[0063] The page segmentation and classification means 40 
may also include image processing means 48 for processing 
the image data after each of the pixels has been labeled with 
an image type and as belonging to a particular window 

[0064] A page segmentation and classification apparatus 
embodying the invention might include a typical computer 
processor and software designed to accomplish each of the 
steps of the two pass method. The apparatus might also 
include image data obtaining means 36 for obtaining an 
image to be processed by the two pass method. The image 
data obtaining means 36 could include a scanner or a device 
for reading a stored image from a memory. The device might 
also include image data generation means 38 for generating 
image data to be segmented and classified by the two pass 
method. The image data generation means could include a 
software program for generating an image or a word pro- 
cessing program that generates a page of text or a page of 
mixed text and images. 

[0065] It is to be appreciated that the outputs of the module 
200 and module 202 are combined — as those skilled in the 
art will appreciate — in combining module 204 to output data 
that takes the form of four layered segmentation data. In this 
regard, the data may be stored in four separate files, one file 
for each layer, or a single file having four identifiable 
portions. Once the data is in this format, it may be used 
advantageously and selectively by the system for a variety of 
desired purposes that can be implemented in the system. 

[0066] The output data is in modified form, as compared 
to the original image data. In certain circumstances, the 
modified data is of optimal form for particular applications. 
Preferably, the output data includes a background layer, a 
foreground layer, and a selector layer, obtained through the 
implementation of the block or object based method 
described above, and a rendering hints layer obtained 
through the implementation of the pixel based method 
described above. As to the pixel-based data, it preferably 
includes information (e.g. statistics) on each pixel type as 
well as information on the window corresponding to each 
pixel. 

[0067] As an alternative, windowed objects identified by 
the pixel based segmentation process may be placed in 
separate partial planes so they can be readily extracted for 
editing or other suitable purposes. 

[0068] Referring to FIG. 8, a method according to the 
present invention is illustrated. The method 500 is initiated 
by obtaining image data (step 502). Initially, obtaining the 
image data may be accomplished using the scanning tech- 
niques discussed above (e.g. via scanner 12) as well as other 
well-known techniques (e.g. in the system of FIG. 2) in 
combination with other image processing and control pro- 



cedures, such techniques and procedures typically being a 
function of the implementation. This image data is then 
input to a first segmentation module (step 504). Preferably, 
the first image segmentation module is a block or object 
based segmentation module 200 that is capable of generating 
MRC data representations. The image data is then input to 
a second image segmentation module (step 506). The second 
image segmentation module preferably takes the form of a 
pixel-based image segmentation module 202 that is opera- 
tive to generate rendering hints that are useful in rendering 
the original image. In the preferred embodiment, the image 
data is input to both the first and second image segmentation 
modules concurrently or simultaneously. 

[0069] The first image segmentation module then gener- 
ates first segmentation data (step 508). The first image 
segmentation data preferably is MRC data that representa- 
tively includes layers of background data, foreground data, 
and selector data. Likewise, the second image segmentation 
module generates second segmentation data (step 510). As 
alluded to above, the second image segmentation data com- 
prises rendering hints. These hints may take a variety of 
forms, but in the preferred embodiment, the rendering hints 
include statistics on the pixels and information on the 
corresponding windows — all such information tending to 
identify characteristics about the pixel such as whether such 
pixels are edge pixels or half-tone pixels. Preferably, these 
two tasks of generating data are accomplished concurrently. 
This could occur whether or not the image data is concur- 
rently input to the respective segmentation modules. 

[0070] Once steps 508 and 510 are complete, the first 
segmentation data is integrated with the second segmenta- 
tion data (step 512). The integration of these two types of 
data is essentially a summing of the data forms. The resultant 
output data thus includes four total layers, three generated by 
the first segmentation module and one generated by the 
second segmentation module. That is, it includes a back- 
ground layer, a foreground layer, and a selector layer — 
obtained through the implementation of the block or object 
based method described above — and a rendering hints layer 
representing the image data, or rendering hints — obtained 
through the implementation of the pixel based method 
described above. This summation of data may be accom- 
plished using any suitable technique. Of course, the manner 
of integration may well be a function of the manner in which 
the output data is maintained and stored. In this regard, the 
four layers may be stored in separate files or in separate 
buffers. Likewise, the data may be stored in a single file or 
on a single storage medium having suitable demarcation 
capabilities. 

[0071] It is important to note that some common cases can 
be specified in advance as "defaults", thus the image rep- 
resentation can be simplified. For example, in common 
documents the text is black. We can specify the default 
foreground to be black, and if the foreground layer is 
missing, it is understood to be black. In such cases, the 
foreground layer does not need to be generated and stored, 
thus simplifying the process and reduce storage space. 
Likewise, a common type of image is text of different colors 
on a white background. We can specify the default back- 
ground to be white, and if the background layer is missing, 
it is understood to be white. In such cases, the apparent 
number of layers of the image representation can be reduced. 
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[0072] Last, the modified image data obtained as a result 
of the integration is output (step 514). Typically, image data 
is segmented most advantageously for the purposes of 
conducting data compression. Once compressed, the data is 
stored, transmitted and/or used for image rendering. In the 
case where the data is transmitted, the modified output data 
must be decompressed at the receiver or destination thereof. 
In the case of image rendering, the data is preferably 
decompressed prior to being provided to the print engine. 

[0073] It should be appreciated that the present invention 
may be implemented using a variety of suitable hardware 
and software techniques, many of which are referenced 
above. However, any such suitable implementation tech- 
niques should be apparent to those skilled in the art upon a 
reading of the present description. 

[0074] The above description merely provides a disclosure 
of particular embodiments of the invention and is not 
intended for the purposes of limiting the same thereto. As 
such, the invention is not limited to only the above-described 
embodiments. Rather, it is recognized that one skilled in the 
art could conceive alternative embodiments that fall within 
the scope of the invention. 

Having thus described the invention, we hereby claim: 

1. A method for segmenting an image, the image com- 
prising pixels and being represented by image data, the 
method comprising steps of: 

obtaining image data; 

inputting the image data into a first image segmentation 
module; 

generating first segmentation data by the first image 
segmentation module, the first image segmentation data 
representing at least one first characteristic of the image 
data; 

inputting the image data into a second image segmenta- 
tion module; 

generating second image segmentation data by the second 
image segmentation module, the second image seg- 
mentation data representing at least one second char- 
acteristic of the image data; and, 

integrating the first image segmentation data with the 
second image segmentation data to obtain modified 
image data. 

2. The method as set forth in claim 1 wherein the inputting 
of the image data to the first image segmentation module and 
the inputting of the image data to the second image seg- 
mentation module are accomplished concurrently. 

3. The method as set forth in claim 1 wherein the 
generating of the first image segmentation data comprises 
generating first characteristic data representing a back- 
ground layer, a selector layer and a foreground layer of the 
image data. 

4. The method as set forth in claim 1 wherein the 
generating of the second image segmentation data comprises 
generating second characteristic data representing rendering 
hints. 

5. The method as set forth in claim 1 wherein the 
integrating of the first and second image segmentation data 
comprises generating the modified image data such that it 
comprises a background layer, a selector layer, and a fore- 



ground layer of the image data and a layer of information 
representing rendering hint for the image data. 

6. A system for segmenting an image, the image com- 
prising pixels and being represented by image data, the 
system comprising: 

means for obtaining image data; 

means for generating first segmentation data, the first 
image segmentation data representing at least one first 
characteristic of the image data; 

means for generating second image segmentation data, 
the second image segmentation data representing at 
least one second characteristic of the image data; and, 

means for integrating the first image segmentation data 
with the second image segmentation data to obtain 
modified image data. 

7. The system as set forth in claim 6 further comprising 
means for inputting the image data to the first image 
segmentation module and inputting the image data to the 
second image segmentation module concurrently. 

8. The system as set forth in claim 6 wherein the first 
image segmentation data comprises first characteristic data 
representing a background layer, a selector layer and a 
foreground layer of the image data. 

9. The system as set forth in claim 6 wherein the second 
image segmentation data comprises second characteristic 
data representing rendering hints. 

10. The system as set forth in claim 6 wherein the 
modified image data comprises a background layer, a selec- 
tor layer, and a foreground layer of the image data and a 
layer of information representing rendering hint for the 
image data. 

11. The system as set forth in claim 6 wherein the means 
for generating the first segmentation data comprises a first 
segmentation module. 

12. The system as set forth in claim 6 wherein the means 
for generating the second image segmentation data com- 
prises a second image segmentation module. 

13. The system as set forth in claim 11 wherein the first 
segmentation module comprises a block based image seg- 
mentation module. 

14. The system as set forth in claim 11 wherein the first 
segmentation module comprises an object based image 
segmentation module. 

15. The system as set forth in claim 12 wherein the second 
image segmentation module comprises a pixel based image 
segmentation module. 

16. An image rendering system adapted for segmenting an 
image, the image comprising pixels and being represented 
by image data, the system comprising: 

a scanner operative to obtain image data; 

a bitmap generator operative to generate a bitmap corre- 
sponding to the image data; 

a first image segmentation module operative to generate 
first image segmentation data, the first image segmen- 
tation data representing at least one first characteristic 
of the image data; 

a second image segmentation module operative to gener- 
ate second image segmentation data, the second image 
segmentation data representing at least one second 
characteristic of the image data; 
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a combining module operative to combine the first image 
segmentation data with the second image segmentation 
data to obtain modified image data; 

a compression module operative to compress the modified 
image data; 

a storage module operative to store the compressed image 
data; 

a decompression module operative to decompress the 
stored image data; and, 

a print engine operative to render the image based on the 
decompressed data. 

17. The system as set forth in claim 16 wherein the system 
is included in a xerographic printing environment. 

18. The system as set forth in claim 16 wherein the print 
engine is a xerographic print engine. 

19. The system as set forth in claim 16 wherein the first 
image segmentation data comprises first characteristic data 



representing a background layer, a selector layer and a 
foreground layer of the image data. 

20. The system as set forth in claim 16 wherein the second 
image segmentation data comprises second characteristic 
data representing rendering hints. 

21. The system as set forth in claim 16 wherein the first 
and second segmentation data comprises layers whereby an 
absence of a selected layer establishes a default position for 
a characteristic represented by the selected layer. 

22. The system as set forth in claim 16 further comprising 
a third image segmentation module operative to generate 
third image segmentation data. 

23. The system as set forth in claim 22 wherein the 
combining module is operative to combine the third image 
segmentation data with the first and second image segmen- 
tation data. 

*T* 
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a combining module operative to combine the first image 
segmentation data with the second image segmentation 
data to obtain modified image data; 

a compression module operative to compress the modified 
image data; 

a storage module operative to store the compressed image 
data; 

a decompression module operative to decompress the 
stored image data; and, 

a print engine operative to render the image based on the 
decompressed data. 

17. The system as set forth in claim 16 wherein the system 
is included in a xerographic printing environment. 

18. The system as set forth in claim 16 wherein the print 
engine is a xerographic print engine. 

19. The system as set forth in claim 16 wherein the first 
image segmentation data comprises first characteristic data 



representing a background layer, a selector layer and a 
foreground layer of the image data. 

20. The system as set forth in claim 16 wherein the second 
image segmentation data comprises second characteristic 
data representing rendering hints. 

21. The system as set forth in claim 16 wherein the first 
and second segmentation data comprises layers whereby an 
absence of a selected layer establishes a default position for 
a characteristic represented by the selected layer. 

22. The system as set forth in claim 16 further comprising 
a third image segmentation module operative to generate 
third image segmentation data. 

23. The system as set forth in claim 22 wherein the 
combining module is operative to combine the third image 
segmentation data with the first and second image segmen- 
tation data. 
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